Building vs. Buying a Computer Vision Platform
“You could do what Roboflow does yourself but…why would you?”
-Jack Clark, Co-Founder of Anthropic, former Policy Directory at OpenAI,
It’s no secret that building a computer vision model on your own is hard work. It requires wrangling together different platforms, open-source tools, and developer notebooks to create a functioning machine learning pipeline. And that’s just the beginning; after you have a trained model you still have to do the engineering work to deploy and maintain it in the wild.
Here at Roboflow, we speak with developers every day who have spent countless hours crafting these workflows - and while they are undoubtedly impressive, they are also labor-intensive and require frequent maintenance. What is worse, when any of these disparate systems break (and they will), it can be impossible to find the source of the error. Especially after the engineer who cobbled it together has left the company.
As a result, these companies spend months of precious developer time (that could have been better focused on the company’s core business problems) creating an unmaintainable monstrosity that is neither scalable nor repeatable.
If you ask us, companies should instead use a toolchain that is crafted and maintained by a team of single-minded experts, and has already been battle tested by thousands of developers. After all, as many readers of this blog will already know, the process isn’t necessarily a simple one. It requires at least six key components:
- Upload Images
- Annotate Images
- Generate Dataset Versions
- Train a Model
- Deploy the Model
- Optimize Performance (Active Learning)
When we built Roboflow, we set out to create an end-to-end solution that captures and streamlines every step of this process. We’re building the computer vision infrastructure so you can focus on your business’ unique differentiators.
Developers are (by nature) resourceful, solution-oriented and adaptable. We’re developers, too - and we get that it can feel counterintuitive to purchase a product like ours when you believe you could build the very same solution yourself. Let’s dig in on some of the common problems computer vision teams run into when they build their own (internal) machine learning tools, and how working with Roboflow can address, alleviate or entirely eliminate this pain.
Problem #1: Versioning and Collaboration
One challenge teams commonly run into if they roll their own computer vision pipeline is that different data scientists and engineers will be working on different versions of the same dataset, and it’s very hard to keep everyone’s changes and additions straight. If one of your team members notices and fixes an error in your annotations, how will that fix make its way into the other team members’ models? If you want to add new training data to improve performance on an edge case, how is that information disseminated? And if a data scientist trains a new model, how do you keep track of how it performed compared to all of your other data scientists’ previous models?
Roboflow enables developers and domain experts alike to create accurate, instantly available computer vision models - and everyone can track the efficacy of each version over time. You can easily view all previous datasets inside your account, and it is clear which version is pre-trained, trained, and deployed. Teams on Roboflow can then compare each of their unique dataset parameters by evaluating the resulting model performance (mAP, Precision and Recall) within their accounts.
Problem #2: Lack of Centrality
Generating and maintaining an externally developed machine learning pipeline often falls to just one individual, or a small team of developers. This means that any amount of employee churn could significantly set back your team’s progress, as everyone else tries to understand (and piece back together) the many different tools and solutions that were used to develop the pipeline. Who has access to which platforms? Where are the annotations stored? What systems need to be manually updated or maintained, and at what frequency?
Roboflow lives in the cloud, which means that everyone, including non-technical stakeholders, will always have access to the entire computer vision workflow. Shared accounts (called “Workspaces”) house all source images, annotations, datasets and trained models (both ready to be deployed, and actively in-deployment). These are the components that make up your entire machine learning pipeline, and they exist in one central repository.
Problem #3: Slow Training Process
Training a model from scratch takes a long time. You don’t want to have to wait for a full model to train every time you make an improvement to your dataset - you want your models to start from a previous checkpoint that encapsulates all of the things your model has previously learned.
This is called Transfer Learning, and it’s baked-into your Roboflow account. Simply choose the trained model you’d like the new model to learn from, and the next iteration will start learning about your new data from where the last one left off.
Problem #4: Poor Model Performance
After all this time and effort has been put into creating your own computer vision pipeline, it can be devastating when the resulting model doesn’t perform as well as your team had hoped or expected. Without the analytics and reporting capabilities you’d need to understand where and how the model is failing, optimizing that model can feel like an expensive and time-consuming game of guess-and-check.
The process of understanding model performance in the wild, and then feeding that inference data back into your source dataset for the next model iteration is called Active Learning, and like Transfer Learning, it is similarly baked-into your Roboflow account. Moreover, our Upload API enables you to pre-select the highest leverage images for this effort, including both low-confidence predictions and high-confidence predictions that are ultimately incorrect.
We’ve designed this workflow to be turnkey for our customers who are actively collecting inference data using one or more models in deployment so their models can get better and better the more they’re used.
Problem #5: Deployment
A trained model isn’t very useful if it can’t be deployed in the wild. With internal tools and custom-made training pipelines, deployment can present a whole host of challenges, depending on the method in which you’d like your model to run predictions on real-world data (e.g. flying a drone to collect aerial imagery, installing a camera in a warehouse, or embedding the model into a smartphone app). Developers can spend weeks building their own API calls and Dockers to unique devices, but these solutions are static and cannot evolve with the rapidly changing landscape of this industry.
There are a number of ways you can deploy a trained model from your Roboflow account, including a hosted inference API, on-device deployment (to targets like NVIDIA Jetson and OpenCV AI Kit devices), and even a web browser. We offer Dockers and documentation to make this process easy and scalable, even as changes occur across the industry and as new generations of tools are released to the public. You’ll be able to get to production in days, not months.
Problem #6: Blocked Initiatives
Teams get blocked. It happens all the time, and especially in the world of computer vision, where new technologies (and advancements to old technologies) often introduce more questions than they answer. In this instance, experts turn to other experts, forums on GitHub, academic papers and even YouTube videos. Wouldn’t it be nice to have a team of machine learning specialists on-call?
One of the best parts of using Roboflow is the educational component that our customers receive from our team directly, the online documentation, and our tutorial videos and blog posts that address common pitfalls, how-to’s and best-practices. When you build with Roboflow, you are never building alone.
With the expertise and help of Roboflow’s experts and your existing engineering talent, you won’t need to hire an expensive team of machine learning PhDs. And if you already have computer vision experts, they can spend their time on the higher leverage, differentiated parts of their jobs rather than reinventing the wheel.
Problem #7: Cost
Many technologists find our solution when they are in the middle of building a robust and complex pipeline that they’ve clearly invested a significant amount of time and resources to develop. We want to emphasize that these efforts are not wasted by switching to Roboflow’s module for better scalability and growth. And the total cost over the long-term will likely be far lower with Roboflow than maintaining a team to support and expand your existing solution.
In fact, the external system you’ve developed can tell us a lot about how to serve you. We can uncover the pros and cons of your current workflow to better meet your project’s unique needs. Our goal is to create a platform that gives you the best of both worlds: a system you can own, touch and manage, and a central hub for all of your data and models, that automates much of the manual effort it takes today to build from scratch.
Ready, Set, Bake-Off!
Want to definitely know whether Roboflow can perform as well as (if not better than) your existing, pre-built pipeline? We challenge you to a bake off. Contact us to receive a free training credit, and we’ll walk you through the process of uploading your images + annotations and training a new model on Roboflow. Let’s compare results and help you make a formal decision about whether to build with us, or build on your own.