Happy Cinco de Mayo! In honor of the holiday, we created and trained a multi-label classification dataset to detect some favorite edibles used to celebrate the holiday here in the United States.
Because we know it’s not just about tacos and beer, we also want to briefly share a bit of info about the holiday to other users. Cinco de Mayo, translated and celebrated yearly on May 5th, commemorates the anniversary of Mexico's victory over the French Empire at the Battle of Puebla in 1862. Though commonly confused for Mexico's Independence Day, these are actually two separate holidays commemorating different events in Mexico’s history.
Now, time to get started on our computer vision model!
What is Multi-Label Classification
Multi-label classification lets your model learn and predict multiple classes for a given image instead of just one (like with single label classification). Multi-label classification is used best when identifying things that are difficult to annotate through bounding boxes like a setting of night, day, party, wedding, funeral, type of art, etc., or when you’re only worried about identifying the object, rather than its absolute location.
Though our dataset could be a bounding box object detection or even instance segmentation project, we chose multi-label classification for this tutorial.
Visit the Roboflow Knowledge Base to learn more about how to create your first computer and machine vision projects on Roboflow.
Using Multiple Datasets
In order to be able to grab enough images of tacos, beer, and null (images we want our model to recognize that what we’re looking for is not there), we used several different datasets to upload on Roboflow. We also made sure to make our dataset diverse with these sources in order to create a dataset that does not overfit due to a lack of diversity. Many null images were also included! We marked these as a “null” class.
Our chosen dataset class names:
- “tacos”
- “beer”
Datasets that were used for this model:
- 200 Bottles (from Roboflow Universe)
- Taco bell images from dinner that day
- Random google images
Check out our guides on how to upload classification datasets, and converting object detection projects to classification projects.
Testing the Model
First, we test the model with Roboflow’s new Roboflow Inference Widget, or rfWidget, tool.
rfWidget makes it easy to perform inference tests so we can visualize how our model may perform in production when it is fully deployed.
To get started, just drag and drop an image onto the rfWidget, or click “browse your device” to select a file from your computer. The rfWidget accepts image files, in formats such as JPEG, PNG, and WEBP.
We can also perform inference with Roboflow’s Hosted Inference API.
Some of you may be wondering, “what’s with all these JSON values coming back from the API and widget, I was expecting advanced code for the detections?”
The secret is, those JSON, Javascript Object Notation, values are the very outputs you receive when performing inference with a deployed model on your webcam, Jetson, Luxonis OAK, Google Coral, Raspberry Pi, or server(s).
Roboflow Multi-Label Classification (MLC) JSON outputs:
- “predictions” is the “object,” or JSON collection, that includes our confidence scores for each class that was present in the dataset during model training [string].
- each individual class label and its associated confidence score is included.
- “confidence” is the prediction’s confidence value [float].
- “predicted_classes” is used as the output for the object or image label(s) of the top predicted classes [string].
Example Roboflow MLC inference output:
{
"predictions": {
"beer": {
"confidence": 0.5030962824821472
},
"taco": {
"confidence": 0.5743188858032227
}
},
"predicted_classes": [
"beer",
"taco"
]
}
The JSON format is flexible enough to be usable in code with not only Javascript, but other programming languages, such as Python. The formatting also makes it possible to easily interact with API’s and for database storage and data retrieval.
And if you’re looking for more project ideas, check out our dataset search feature on Roboflow Universe.
Have fun, stay safe, drink responsibly!