Flaw Detection Using Computer Vision

Abirami Vina -

From double-checking components on an electronic circuit board to looking for dents and scratches on cars, there are many innovative computer vision applications related to inspection and quality control. Computer vision is a subfield of artificial intelligence (AI) that gives computers the ability to handle tasks like recognizing and locating anomalies within images or videos.

Until recently, flaw detection tasks were mostly manual and depended on human judgment. Manually detecting flaws isn’t 100% effective, and each visual inspector may judge flaws differently, making the process difficult to scale.

Scalability is crucial, especially since flaw detection systems are widely used across various industries. For example, in manufacturing, it can automatically identify and classify different defects like cracks and scratches. In the automotive industry, it is used to check components like car bodies, engines, and tires for defects that could compromise safety.

In this article, we’ll explore the concept of using computer vision for flaw detection and discuss its importance in various applications. We’ll also guide you through a coding example that showcases how to detect flaws in glass bottles using a pre-trained model from Roboflow Universe.

Understanding Flaw Detection

You can identify small product flaws that can go unnoticed in the supply chain using different flaw detection methods. Such defects can cause structural and functional issues in a product that can reduce its quality and harm a brand name’s reputation. For instance, in automobile manufacturing, parts like bearings and gear chains have small and complex parts that need to be thoroughly inspected to make sure they meet the safety standards of the vehicles they are fitted into.

Manual inspection of an automobile’s gearbox. (Source)

Flaw detection can be used to detect a wide range of defects, including surface defects, structural issues, and material inconsistencies. Surface defects, such as scratches, dents, or blemishes, are visible on the product's exterior. On the other hand, structural issues, like cracks or delaminations, may be internal and require more advanced inspection techniques. A good example is in aviation; external cracks or dents can affect the aerodynamics of a plane. Similarly, material inconsistencies can be color, texture, or composition variations that compromise product quality and performance.

There are many ways to perform flaw detection - what makes computer vision stand out? Computer vision systems can quickly process large amounts of data and shorten inspection times. They are also less susceptible to human error and can result in greater accuracy and consistency in flaw identification. Automated detection also makes real-time monitoring and analysis possible. Manufacturers can take immediate corrective actions as needed.

Using Computer Vision for Flaw Detection

Computer vision models are trained on vast datasets of images containing both defective and non-defective products. By learning to recognize patterns and features associated with different types of defects, these models can accurately pinpoint and categorize flaws in real-time. These models are also great for handling complicated scenarios, like detecting defects on products in a moving assembly line. They can also be fine-tuned for application-specific product defects.

There are many pre-trained computer vision models available online that you can use for your applications. You can also train your own custom model by creating a dataset of images (of defects) that is specific to your application. In this article, we’ll be using one of Roboflow’s pre-trained models to detect flaws in glass bottles for our coding example in the next section.

Roboflow model that can detect flaws in glass bottles. (Source)

Using Object Detection To Detect Flaws

Now that we’ve understood what flaw detection is and how computer vision can be used to automate and speed up the process, let’s use a pre-trained object detection model to check for flaws in glass bottles. Here, we’ll be exploring how to apply an object detection model, and we won’t go into the details of custom training a model to detect various types of defects and flaws. To learn more about training a custom object detection model, check out our YOLOv8 custom training guide.

The pre-trained model we’ll be using has been trained to detect flaws on the surface of glass bottles. The model is available on Roboflow Universe. Roboflow Universe is a computer vision community that provides access to open-source datasets and pre-trained models. It boasts a collection of over 200,000 datasets and 50,000 pre-trained models.

To get started, sign up for a Roboflow account and navigate to the model’s deployment page. If you scroll down, you’ll see a piece of sample code that shows how to deploy the API for this model, as shown below.

To try using this model, you’ll need an image to run inferences on. We’ve used an image downloaded from the dataset associated with the trained model. You can use the same image, your own image, or download an image from the internet.

An image of a glass bottle that we’ll be using. (Source)

Step #1: Setting Up the Environment

First, install the inference library, which provides easy access to pre-trained models. Roboflow Inference is a tool that can help you deploy these models directly into your applications. You can install it using pip with the following command:

pip install inference-sdk

Step #2: Loading the Model & Running the Inference

After installing the inference library, the next step is to initialize the inference client with your API URL and your API key. Replace "path/to/image" with the path to your input image and  "Your API Key" with your Roboflow API key. You can refer to the Roboflow documentation for more instructions on how to retrieve your API key.

from inference_sdk import InferenceHTTPClient
import supervision as sv
import cv2

# Initialize the inference client with the API URL and API key
CLIENT = InferenceHTTPClient(
    api_url="https://detect.roboflow.com",
    api_key="Your API Key"
)

# Specify the path to the image to be analyzed
img_path = "path/to/image"

# Perform inference using the specified model
result = CLIENT.infer(img_path, model_id="glass-defect-detection-fvbcu/2")

If you want to print out and view the results of running inference for inspection, add the following command:

print(result)

Step #3: Extract the Results and Visualize the Output

The following code will display the output image by drawing bounding boxes around the detections with labels based on the predictions of the model analysis.

# Extract the class labels from the inference results
labels = [item["class"] for item in result["predictions"]]

# Convert the inference results to a format compatible with Supervision (sv) for further processing
detections = sv.Detections.from_inference(result)

# Initialize the annotators: one for labels and one for bounding boxes
label_annotator = sv.LabelAnnotator()
box_annotator = sv.BoxAnnotator()

# Read the input image using OpenCV
image = cv2.imread(img_path)

# Annotate the image with bounding boxes
annotated_image = box_annotator.annotate(
    scene=image, detections=detections
)

# Annotate the image with labels (class names) above the bounding boxes
annotated_image = label_annotator.annotate(
    scene=annotated_image, detections=detections, labels=labels
)

# Display the annotated image using Supervision's plot function
sv.plot_image(image=annotated_image, size=(16, 16))

If you include the code given below, the output image will be saved as a separate file named "Result.jpg."

cv2.imwrite("Result.jpg", annotated_image)

Here is the output image. As you can see, there is a flaw in the glass bottle’s surface.

Flaw Detection in Manufacturing

Defect detection systems in the manufacturing industry can cut costs by finding issues early, reducing waste, and making employees more productive. It also helps prevent dips in sales, waste products, legal problems, and extra costs for fixing or replacing products. By using computer vision to inspect products, manufacturers can work faster, spend less on training employees, and make fewer mistakes.

Blister pack defect inspection system. (Source)

Traditional methods of checking car parts for defects can be slow and often lead to mistakes. In the car manufacturing industry, vision inspection systems are becoming more common. They are used to check paint, weld quality, assembly, parts, and quality control. In these systems, high-resolution cameras take pictures. 

Flaw detection in the automotive industry. (Source)

Identifying Defects in Electronics

Another industry that is taking advantage of computer vision is electronics manufacturing. Computer vision is a great tool for finding flaws in complex electronics. It can automatically detect and classify problems like cracks, scratches, and missing components.

Computer vision systems are faster, more accurate, and give consistent results compared to manual checking (which is done by hand). It can also detect soldering issues and spot when components have been placed incorrectly on circuit boards. Along the same lines, technicians can use computer vision to find faults and defects while performing repairs and maintenance.

Using object detection to check that all components of a PCB are in place. (Source)

Conclusion

Computer vision is an excellent tool for detecting flaws in products. It outperforms manual inspection by being faster, more accurate, and consistently reliable. Industries across the board are using this technology to guarantee that their products meet high quality and safety standards.

You can use object detection models fine-tuned on a specific use case (i.e. glass chip detection, PCB flaw detection) to identify issues that can be flagged in a quality assurance process.

In this guide, we demonstrated an example of using a glass defect detection model to identify flaws. Such a model could be integrated with automated assembly line systems to send defective products to a human for review, or to discard them and reuse materials to make a new product if possible.

Want to Learn More?

Check out these resources to keep exploring: