This project creates a system to automatically detect damaged pills, foreign matter, and incorrect pills during a manufacturing process. Wrong or damaged pills making their way to consumers can lead to health hazards, recalls, loss of sales, and reputation damage.
For this tutorial, we will focus on the final examination of pills during the quality inspection process before they are sent for packaging. Visual inspections can be done at very high speeds and require high accuracy.
The architecture of such a system is proposed in the following diagram:
A computer vision model can be trained on a cloud server, and the model is then deployed on a controller that controls the conveyor belt for inspecting pills. The controller gets the data from a mounted camera and then sorts the pills depending on their level of quality. The correct pills are sent for packaging, damaged pills are discarded, and the data is stored back to the server for reporting and research.
This type of machine is not only useful for damage inspection but also for sorting by pill type before packaging. Prior to being placed into packaging, pills go through visual inspection to ensure features such as dimensions, surface texture, shape, and color meet specifications.
Let's get started building the system!
Project Requirements
To build this project as outlined above, the following pieces of equipment are needed:
- Raspberry Pi 4 or any Linux Compatible board
- USB Webcam
- USB Keyboard
- WiFi internet connectivity
Pill Dataset Collection and Preparation
This project uses two classes: ‘damaged-pills’ and ‘foreign-matter’. The public project pill-inspection-od is available at Roboflow Universe for you to open and explore alongside this guide. The following figure shows example images from the dataset:
All images are uploaded to Roboflow platform and labeled corresponding to their class type.
Once all the images are labeled, the following pre-processing and augmentations are applied and the dataset version is generated.
Pill Inspection Model Training
Once the dataset has been generated, Roboflow Train is used to train the model.
The ‘Training from Scratch’ option was chosen which selects the Roboflow 2.0 Object Detection (Accurate) model and trains the model from scratch.
Click the 'Start Training' button in the UI to begin training the model using Roboflow once the dataset version is generated.
In the next dialog box choose your preferred model type.
Next, select ‘Train from Scratch’.
Roboflow will automatically prepare the dataset for training.
The training will begin automatically and you can observe your model training in real-time.
The model is then hosted and available to be used for deployment via API and to edge devices.
The following graphs show how the model performed during the training cycle. Using these graphs it is quite easy to understand the model is learning from the input data.
Testing Model Predictions
The following images show how the model performs on test data. From the images we can see that the model is predicting broken or damaged pills with a high accuracy which is needed for such a system.
These images depict the ground truth vs the model prediction, and it is clearly seen that the model detects broken pills very well.
You can use the Deploy page in Roboflow to test the model directly in your browser using images, videos, and webcams to understand model performance.
Pill Inspection System Deployment
This model can be deployed for different machines using the Robolow API and the below architecture was used in this project:
Deployment utilized the python script on Raspberry Pi for inferencing.
Conclusion
We’ve just completed a step-by-step process for creating a pill inspection computer vision model, deploying the model to an edge device, and using the edge device in a larger system to automate removing damaged pills from being shipped.
This same architecture can be used in any manufacturing setting to automate quality assurance processes and add intelligent augmentation for workers in a manufacturing environment.
Cite this Post
Use the following entry to cite this post in your research:
Timothy Malche. (Feb 24, 2023). Building a Computer Vision Assisted Pill Inspection System. Roboflow Blog: https://blog.roboflow.com/computer-vision-pill-inspection-quality-assurance/
Discuss this Post
If you have any questions about this blog post, start a discussion on the Roboflow Forum.