When you are building computer vision models, it can be helpful to regularly gather new data for use in training future versions of your models. You can gather data from your application in real time, and use predictions from your model as labels for a new training dataset.
This technique, called active learning, can help you improve your model performance over time by helping you gather more representative data for use in training new model versions. Active learning can also help prevent data drift, where your model becomes less effective as the real world conditions in which your model is deployed change.
You can also use active learning with an existing, public model to auto-label data for use in training a new model.
In this guide, we are going to discuss how to build an active learning workflow with Roboflow Workflows. Roboflow Workflows is a web application where you can build computer vision workflows with a visual editor.
Here is a demo of our Workflow running:
Without further ado, let’s get started!
What is Roboflow Workflows?
Workflows is a computer vision application builder. With Workflows, you can build complex computer vision applications in a web interface and then deploy Workflows with the Roboflow cloud, on an edge device such as an NVIDIA Jetson, or on any cloud provider like GCP or AWS.
Workflows has a wide range of pre-built functions available for use in your projects and you can test your pipeline in the Workflows web editor, allowing you to iteratively develop logic without writing any code.
Build an Active Learning Pipeline for Computer Vision
In this guide, we will build a workflow that improves a yard management model. When the model runs, successful predictions from our model will be added back to our dataset alongside the accompanying image. We can then use that data to train future model versions.
Our workflow will run a yard management model that detects shipping containers, chassis IDs, and other features of intermodal containers. The workflow will selectively sample images to send back to our dataset.
Here is what our final workflow will look like:
Step #1: Create a New Workflow
To get started, first you will need a model in Roboflow. Follow the Roboflow Getting Started guide to learn how to create a project and train a model in Roboflow.
Once you have a model, navigate to Roboflow Workflows by clicking “Workflows” in the sidebar of your Roboflow dashboard.
Click the “Create Workflow” button to create a new Workflow:
A few example workflows will appear. Select the “Smart Sampling” workflow, then click “Create Workflow”.
You will be taken to the Roboflow Workflows editor where you can customize your workflow.
Step #2: Configure your Model
Our workflow accepts an image (or frame from a video), runs an object detection model, and returns a response. Our workflow also saves some images that have been run through the workflow back to our dataset.
We need to configure what model we want to use in our workflow.
To do this, click on the “Object Detection Model” block and set a model:
You can select any model in your workspace, or any of the 50,000+ open models available on Roboflow Universe.
Using a model on Roboflow Universe is ideal if you want to bootstrap creating a new model. For example, you may see a model that is good at a particular task (i.e. detecting wood defects) and use that to help label data for your own personal project.
For this guide, we will use a model in our workspace, since we already have a model we want to improve.
Click on the model you want to use, then click “Select This Model”.
Step #3: Configure a Dataset
Next, we need to configure our active learning setup. This is called “Roboflow Dataset Upload” in the Roboflow Workflows builder.
To set up active learning, we need a dataset in which images should be saved with predictions returned by our model. This is called a “Target Project”.
If you are using active learning to improve a model you have already trained, your Target Project may be the same as the model you are using. If you are using a public or zero-shot model (i.e. YOLO World) to auto-label data, you may use a new project where you are storing images for your own model/
To set a Target Project, click on the Roboflow Dataset Upload block then choose a project from the Target Project dropdown.
For this example, we are using the same dataset as the one associated with the yard model we are using, since we want to improve the yard model with data labeled from our workflow.
Step #4: Add a Visualizer
By default, the workflow template we have used does not have a visualization block that shows results from our model. We can add one so that we can see the bounding boxes returned by our model.
Click “Add Block” in the top right corner of the workflow builder interface, then choose “Bounding Box Visualizer”:
When the block is in your workflow, click on it and make sure the image references $inputs.image and the predictions reference $steps.model.predictions:
Then, click on the “Response” block. Click “Add Output” at the bottom of the block and select $steps.bounding_box_visualization.image as the reference. This will let us return the result from our visualizer in the workflow.
We are now ready to test our workflow!
Step #5: Test the Active Learning Workflow
To test a Workflow, click “Run Preview” at the top of the page.
To run a preview, drag and drop an image into the image input field. Then, click “Run Preview” to run your Workflow.
The Workflow will run and provide two outputs:
- A JSON view, with all of the data we configured our Workflow to return, and;
- A visualization of predictions from our model, drawn using the Bounding Box Visualization block that we included in our Workflow.
Let’s run the Workflow on this image of a shipping container.
Here is the Show Visual output from our workflow:
Our model successfully identified a shipping container and the various classes our model supports: the logo, chassis ID, and more.
Go back to your dataset in Roboflow and click on the “Annotate” tab. You should see a new image with the tag “workflows_data_collector”:
This contains the image that was saved by our workflow. Click on the image to see the results from the model:
We can see the annotations from our model, which contains:
- One chassis ID
- One container
- Two IDs
- One logo
We can then add this image to our dataset, then use it to train a future version of our model.
Read the Roboflow documentation to learn more about adding an image to a dataset.
Step #5: Deploying the Workflow
Workflows can be deployed in the Roboflow cloud, or on an edge device such as an NVIDIA Jetson or a Raspberry Pi.
To deploy a Workflow, click “Deploy Workflow” in the Workflow builder. You can then choose how you want to deploy your Workflow.
[add image]
Workflows can be run on individual videos, video files, webcam feeds, or RTSP streams. Instructions for deploying your Workflow, which include your workflow ID, are available in the Deploy Workflow tab in the Workflow tab.
Conclusion
You can use Roboflow Workflows to define active learning pipelines in computer vision deployments.
In this guide, we walked through how to deploy an active learning workflow with Roboflow Workflows, an interactive computer vision application builder. We used a pre-made Smart Sampling workflow and configured it with our own model and dataset. We then tested the workflow and validated that the result from a test was saved in our dataset.
To learn more about building computer vision applications with Workflows, refer to the Roboflow Workflows documentation.