Francesco

17 Jan 2023 • 8 min read

SUMMARY Multi-GPU neural network training requires moving tensors between processes, and PyTorch's torch.distributed module provides six collective communication strategies to handle this: reduce, all reduce, scatter, gather, all gather, and broadcast. This post covers each strategy with code examples and diagrams, explaining when to use each

30 Aug 2022 • 7 min read

COCO is an industry standard dataset for benchmarking the performance of object detection models. The dataset was created by "gathering images of complex everyday scenes containing common objects in their natural context" and contains image annotations in 91 categories, with over 1.5 million object instances. People have

3 Aug 2022 • 8 min read

SUMMARY Managing NVIDIA drivers and CUDA versions directly on a host machine leads to brittle environments that break when dependencies drift. This guide walks through installing NVIDIA drivers on Ubuntu 20.04, setting up Docker and nvidia-docker for GPU-accelerated containers, pulling an official PyTorch image from NVIDIA NGC,