Roboflow now supports semantic segmentation projects end-to-end allowing you to use Roboflow Annotate to label data, Roboflow Train to build models, and Roboflow Deploy for inference.
This release, alongside support of instance segmentation, offers users an additional image segmentation project type to build computer vision pipelines with Roboflow.
What is Semantic Segmentation?
Semantic segmentation is the computer vision task of assigning each pixel in an image to a specific class. This is different from instance segmentation because individual objects are not identified with a bounding box as well as the pixel level mask.
Semantic segmentation models, like SegFormer, are trained to predict the class of each individual pixel and their performance is evaluated by Mean Intersection Over Union where Intersection Over Union = true_positives / (true_positives + false_positives + false_negatives).
Semantic Segmentation Use Cases
Semantic segmentation can be used in a variety of scenarios and is often associated with autonomous vehicles or robotics. Classifying each pixel in an image is best used when there is a need for exact predictions at such a detailed level.
Additional common use cases are reading medical images, background removal in media editing, defect detection in manufacturing, or identifying objects in aerial or satellite imagery.
Labeling for Semantic Segmentation
Whether you're uploading your own data, using a video from Youtube , or cloning data from the 100+ million open source images from Roboflow Universe, you'll need to apply pixel-level labels to your data. Labeling data for semantic segmentation can be time consuming if you don't have the right tools for the job.
AI-assisted labeling tools help speed up the labeling process and make semantic segmentation projects more feasible. To apply an accurate label, you'll need to label each class in the image and using an automated labeling product like the Roboflow Smart Polygon tool makes that fast.
Accurate labeling will improve the performance of your model so investing in tools to make that possible are an important part of your overall semantic segmentation tool chain. When labeling for semantic segmentation, make sure your masks are properly ordered if you're labeling backgrounds and complex scenes.
Once you've labeled your data, applied pre-processing and augmentations, and generated a dataset, you're ready to train a model.
Training Semantic Segmentation Models
Roboflow offers a one-click model training solution, Roboflow Train, which gives you a state of the art model, hosted at an API endpoint, and customized for your dataset.
In order to help improve the performance of your model based on your specific dataset, you can utilize transfer learning from the pre-trained ImageNet model.
AutoML tools are a great way to streamline model training, decrease costs for on-demand GPUs, and reduce the number of tools you need to manage as you work to build your computer vision pipeline.
For anyone interested in training a semantic segmentation model outside of Roboflow, you can follow our tutorial on training SegFormer with a custom dataset.
Inference with Semantic Segmentation Models
Once your model is done training, you'll be able to use the model for inference on your custom data and integrate into your application with a ready to use API endpoint.
Using Roboflow Deploy, your computer vision pipeline is automatically set up to handle demanding workloads with autoscaling infrastructure that includes load balancing, supports burst and no burst, and is always on without any custom engineering work
At this point, you'll be able to have your model running in production and begin evaluating performance to identify how you can improve your model. The Roboflow Python package makes it easy to sample images from production back into Roboflow so you can continue improving your model over time or as edge cases come up, a process known as active learning.
Start a Semantic Segmentation Project
You can build with Roboflow for free at each step in your computer vision pipeline and use any of the 100,000+ open source datasets in Roboflow Universe.
Whether you're interested in mapping sidewalks from aerial imagery, identifying cracks in infrastructure, or measuring brain tumors, the power of semantic segmentation is now at your fingertips and we can't wait to see what you build.
Cite this Post
Use the following entry to cite this post in your research:
Trevor Lynn. (Oct 26, 2022). Launch: Semantic Segmentation for Labeling, Training, Deployment. Roboflow Blog: https://blog.roboflow.com/semantic-segmentation-roboflow/
Discuss this Post
If you have any questions about this blog post, start a discussion on the Roboflow Forum.