How to Use the Detectron2 Model Zoo for Object Detection

What Is Detectron2?

Detectron2 is Facebook AI Research's open source library for detection and segmentation, released in 2019 as a complete PyTorch rewrite of the Caffe-based Detectron. Facebook has used it across its own products, and its modular design made it a standard research platform for two-stage detectors.

Alongside the library, FAIR published the Detectron2 model zoo: architectures with weights pre-trained on COCO, including multiple Faster R-CNN variants and RetinaNet for object detection, Mask R-CNN for instance segmentation, keypoint detection models, and panoptic segmentation models. Each entry lists its box AP, inference speed, and training memory, so you can weigh accuracy against speed before downloading anything.

Learn how to train Detectron2 on custom object detection data.

Using Detectron2 for Object Detection

Detectron2 is installed from source. Follow the installation instructions for your PyTorch and CUDA versions:

pip install 'git+https://github.com/facebookresearch/detectron2.git'

Note that the library's last tagged release shipped in 2021, so expect to resolve version pins yourself on current PyTorch builds. A GPU machine is assumed throughout.

Prepare Your Dataset

Detectron2 reads COCO JSON annotations. The fastest way to get a labeled dataset into that format is Roboflow: upload images, label them with Auto Label and Label Assist, generate a version, and export in COCO JSON format. You can also fork any of the 200,000+ labeled datasets on Roboflow Universe and export the same way.

Register the exported splits with Detectron2:

from detectron2.data.datasets import register_coco_instances

register_coco_instances("my_dataset_train", {}, "train/_annotations.coco.json", "train")
register_coco_instances("my_dataset_val", {}, "valid/_annotations.coco.json", "valid")

Train a Model from the Zoo

The config below trains Faster R-CNN with a ResNeXt-101 backbone, initialized from its zoo checkpoint. The trainer subclass adds COCO evaluation during training:

import os
from detectron2 import model_zoo
from detectron2.config import get_cfg
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator

class CocoTrainer(DefaultTrainer):
    @classmethod
    def build_evaluator(cls, cfg, dataset_name, output_folder=None):
        if output_folder is None:
            output_folder = os.path.join(cfg.OUTPUT_DIR, "eval")
        return COCOEvaluator(dataset_name, cfg, False, output_folder)

cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x.yaml"))
cfg.DATASETS.TRAIN = ("my_dataset_train",)
cfg.DATASETS.TEST = ("my_dataset_val",)

cfg.DATALOADER.NUM_WORKERS = 4
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x.yaml")  # initialize from the zoo checkpoint
cfg.SOLVER.IMS_PER_BATCH = 4
cfg.SOLVER.BASE_LR = 0.001

cfg.SOLVER.WARMUP_ITERS = 1000
cfg.SOLVER.MAX_ITER = 3000  # adjust up if val mAP is still rising, down if overfitting
cfg.SOLVER.STEPS = (1000, 1500)
cfg.SOLVER.GAMMA = 0.05

cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 64
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 13  # the number of classes in your dataset

cfg.TEST.EVAL_PERIOD = 500

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
trainer = CocoTrainer(cfg)
trainer.resume_or_load(resume=False)
trainer.train()

Transfer learning from the COCO checkpoint means the model starts with useful visual features and converges far faster than training from scratch. Watch validation mAP during training: if it is still rising at the end, increase MAX_ITER; if it peaks early and declines, reduce it.

Swap in a Different Model from the Zoo

The zoo's config system makes changing architectures a one-string edit. The two lines that matter are the ones naming the YAML file:

cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x.yaml"))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x.yaml")

To use another detection model:

1. Open the model zoo table and find the COCO Object Detection baselines.
2. Copy the config path for the model you want, for example a RetinaNet or a lighter ResNet-50 Faster R-CNN.
3. Paste that path into both lines above.

Training then starts from the new architecture and its pre-trained checkpoint. Nothing else in the script changes.

A rough guide to choosing: the X101 Faster R-CNN configs give the highest box AP at the slowest speed and highest memory, the R50 FPN variants are the standard accuracy-speed middle ground, and RetinaNet trades some accuracy for single-stage simplicity. The zoo table's box AP and inference-time columns make the comparison concrete for your latency budget.

Evaluate the Result

The CocoTrainer writes COCO-style evaluation to the eval folder each EVAL_PERIOD iterations: mAP at IoU 0.5:0.95, mAP at 0.5, and per-category AP. Per-category numbers are the ones to act on, since a lagging class is usually a data problem. Add and label more examples of that class in Roboflow, re-export, and retrain. This mean average precision explainer covers reading these metrics.

Is Detectron2 still maintained?

The repository remains public and its checkpoints still download, but the last tagged release was in 2021 and development activity has slowed substantially. Budget time for dependency work when installing on current PyTorch versions.

Which model zoo entry should I start with?

The R50 FPN 3x Faster R-CNN config is the standard starting point: solid accuracy, moderate speed, and the most community reference material. Move to an X101 config when accuracy justifies the extra compute, or RetinaNet when you want a single-stage detector.

Can I use Detectron2 model zoo checkpoints commercially?

Detectron2 is released under the Apache 2.0 license. Review the model zoo page for any notes on specific checkpoints, and confirm the license status of any dataset your fine-tuning data comes from.

Does the zoo cover more than object detection?

Yes. Beyond the detection baselines it includes Mask R-CNN configs for instance segmentation, keypoint R-CNN models for pose, and panoptic FPN models, all following the same config-and-checkpoint pattern.

What are the alternatives for a new project?

Transformer-based detectors in the DETR family, such as RF-DETR, now exceed R-CNN-era accuracy at real-time speeds, skip anchor and NMS tuning, and are actively maintained, with hosted training available in Roboflow.

Get Started

Pick a checkpoint from the zoo, point the config at your COCO JSON export, and fine-tune. Roboflow handles the labeling, versioning, and COCO JSON export on the data side, whichever architecture you train.

Cite this Post

Use the following entry to cite this post in your research:

Erik Kokalj. (May 4, 2026). How to Use the Detectron2 Model Zoo for Object Detection. Roboflow Blog: https://blog.roboflow.com/how-to-use-the-detectron2-object-detection-model-zoo/