Launch: Collaborative Annotations

Roboflow Annotate has become an essential tool used by tens of thousands of developers to label images of everything from planes to petri dishes. Since its launch just over a year ago, it has made creating training data for your computer vision models simple and fast.

But there's more to annotation than labeling images. Today we're launching a new workflow that makes managing the pipeline of uploading, assigning, and reviewing easy for large teams working on multiple projects. It takes the tool you know and love and makes it multiplayer by default.

Best of all, most of these new features are available for free to all Roboflow users on our existing plans! If you'd like to add access to advanced annotation workflow management for your team, contact your account rep.

Board View

The board view takes center stage on the new Annotate tab in Roboflow. It gives an at-a-glance view of the current state of your images as they go through the annotation process.

A project's annotation board showing annotation jobs in various states.

Job Assignment

One of the biggest feature requests we've gotten is a way to divide annotation work amongst a team or assign it to specific people responsible for annotation. Splitting the work into jobs assigned to individual team members means you won't have to worry about stepping on each others' work if you're online at the same time.

Splitting a batch of images between two labelers.

Notifications

To go along with task assignment, we've added notifications that alert your team members when there's work assigned to them. This should help streamline the collaboration process.

An email notification received by an annotator when they've been assigned a new job.

Annotator Instructions

It's important that your annotators are all on the same page, so we've added an instructions pane for passing messages to your teammates.

Example instructions on an annotation job, including a link to an external style guide document.

Review Mode

It's important to carefully curate the ground truth that enters your training dataset so we've added an additional "review" step for teams that want more advanced controls to be able to make sure a second set of eyes checks your annotators' work. You can individually approve or reject annotated images and send them back to the annotator for rework when necessary.

By default, the person who assigned the images to be annotated is the Reviewer, but you can choose to reassign this task to someone else.

If you'd like access to the advanced workflow, including Review Mode, reach out to our sales team.

Image Batches

Notably, Roboflow supports uploading images to your image library that you might not intend to label or train on (yet). Sampling real world images your model sees in the wild is a great way to spot edge cases and failure modes. You can decide later on which images you want to annotate and re-train with to most improve your model.

Some example batches of Unannotated images ready to be assigned.

The new Annotation workflow supports Image Batches which are a rudimentary way to store images in different buckets you can pull from later. Batches are just the beginning; we are working on adding powerful tools for organizing and exploring your library of unannotated images.

What's Next

All of this comes together to help you create higher quality image datasets faster, together. This enables you to build better models, create more value for your business, and deliver magical experiences to end-users.

We're already working on the next phase of this product which includes things like dataset rollback, role-based access controls, improved ontology management, annotator assessment, label assist 2.0, and much, much more.

Collaborative annotation was developed in concert with our amazing customers; if you have suggestions for making the product better, share them with us on our forums!