Color Sensing with Computer Vision

Automated color detection is more important in our daily lives than we might realize. For instance, it helps factories make sure products look just right by checking for color differences. In cars, it assists with parking and staying in the correct lane. Even your smartphone uses color detection to make photos look better and apply cool filters.

With computer vision, you can identify the main colors of an object in an image.

In this article, we'll dive into how computer vision techniques can be used to identify colors in an image. We’ll also walk through a coding example that shows how detecting colors can be used in a manufacturing application. Specifically, we’ll look at how to check the color of bottle caps using object detection and the HSV color space.

Understanding Color Spaces

Color spaces are like maps that help computers understand and work with colors. They’re essential for organizing colors in a way that digital systems can process. Two of the most commonly used color spaces in image analysis are RGB (Red, Green, Blue) and HSV (Hue, Saturation, Value). Each has its own unique features and makes them suitable for different tasks.

RGB is perhaps the most well-known color space. It works by mixing different amounts of red, green, and blue light to create a wide range of colors. Each color is made up of these three channels, with each channel having a value between 0 and 255. It gives us millions of possible color combinations, which is why RGB is widely used for digital screens and images. However, while RGB is great for making colors, it doesn’t always match how humans see and distinguish colors. For example, small changes in lighting can cause significant shifts in RGB values, making it tricky for tasks that require precise color recognition.

HSV is another color space that’s more aligned with how we naturally see colors. It breaks down colors into three parts:

• Hue is a type of color (like red, blue, or yellow).
• Saturation tells us how strong or vivid the color is.
• Value shows how bright or dark the color is.

What makes HSV special is that it separates color (hue and saturation) from brightness (value). It is particularly useful in computer vision because it helps the computer recognize colors accurately, even if the lighting changes. For instance, in bright sunlight or shadow, the value might change, but the hue stays relatively consistent. That’s why HSV is often preferred for tasks like identifying specific colors, even when the lighting isn’t perfect.

Using Computer Vision for Color Detection

Using computer vision to detect colors in images or videos is a great way to understand and analyze visual information. One of the best tools for this is OpenCV, a popular library that is packed with algorithms that help with different computer vision techniques.

The HSV color space is used more often to detect colors than the RGB color space. The reason for this is that HSV separates the actual color (hue) from the lightness and darkness (value) and the intensity of the color (saturation). This separation makes it easier and more reliable to detect colors. For instance, by setting specific ranges for hue, saturation, and value, you can tell the computer to pick out certain colors, like all shades of blue in a picture. OpenCV makes this process straightforward by offering functions to convert images from RGB to HSV and then filter out the colors you're interested in.

While using HSV for color detection is helpful, there are situations where you might need something more advanced, like running k-means clustering on a specific region of an image that has been isolated using segmentation. If you are interested in learning more about this technique, refer to our color detection with k-means clustering guide.

Using OpenCV for Color Detection

Let’s explore how to use computer vision to detect bottles in a production line and determine their colors based on the bottle caps. The process involves two main stages: first, detecting the bottles using a trained object detection model available in the Roboflow Universe, and second, performing color detection using the HSV color space.

Step #1: Install Dependencies

To get started, install the inference library, which provides easy access to pre-trained models. Roboflow Inference is a tool that can help you deploy these models directly into your applications. You can install it using pip with the following command:

pip install inference

The following libraries will help us load images, detect bottles, and analyze colors. Make sure to import them before proceeding.

import cv2
import matplotlib.pyplot as plt
from inference_sdk import InferenceHTTPClient
import numpy as np

Step #2: Detect Bottles Using a Pre-trained Roboflow Model

This step will focus on identifying the locations of bottles in an image. First, we need to set up the Roboflow client, which will be used to load the Roboflow model. You can find the model details here.

We can initialize the InferenceHTTPClient by providing the Roboflow API URL and your specific API key. You can refer to the Roboflow documentation for more instructions on how to retrieve your API key. 

# Initialize the Roboflow client
CLIENT = InferenceHTTPClient(
    api_url="https://detect.roboflow.com",
    api_key="YOUR_API_KEY"
)

Next, load the image of the bottles you want to detect. We’ll be using an image from the dataset associated with the model. You can use the same image or your own. Replace "path/to/image" with the path to your input image.

# Load the image
image_path = "path/to/image"
image = cv2.imread(image_path)

Now, we can use the initialized client to detect bottles in the loaded image. The model_id corresponds to a specific model trained to detect bottles. 

# Detect bottles using the Roboflow model
result = CLIENT.infer(image_path, model_id="bottle-detection-bjabk/1")

Next, we can filter detections based on confidence. Each prediction includes coordinates (x, y), size (width, height), and a confidence score. We filter out any detections with confidence below the threshold (0.7 in this case), ensuring only reliable detections are considered.

# Set the minimum confidence threshold
min_confidence = 0.7  # Adjust this value for your needs

# Process each detected bottle
detections = []
for prediction in result['predictions']:
    x = prediction['x']
    y = prediction['y']
    width = prediction['width']
    height = prediction['height']
    confidence = prediction['confidence']

    # Filter detections by confidence
    if confidence >= min_confidence:
        detections.append({
            'x': x,
            'y': y,
            'width': width,
            'height': height,
            'confidence': confidence
        })

Now, visualize the detected bottles by drawing bounding boxes around them. 

# Draw bounding boxes around detected bottles
for detection in detections:
    cv2.rectangle(image, 
                  (int(detection['x'] - detection['width'] / 2), int(detection['y'] - detection['height'] / 2)), 
                  (int(detection['x'] + detection['width'] / 2), int(detection['y'] + detection['height'] / 2)), 
                  (0, 255, 0), 2)

# Display the output
cv2.imshow('Detected Bottles', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Step #3: Perform Color Detection Using the HSV Color Space

Next, we can focus on detecting the color of the bottle caps. The function below focuses on the top 20% of each detected bottle to determine its color.

We convert the region of interest (ROI) to the HSV color space, which makes it easier to differentiate between colors. Then, we create masks for blue, orange, and yellow colors and calculate the percentage of each color in the ROI. The dominant color is determined by comparing these percentages.

def detect_color(image, x, y, width, height):
    # Extract the region of interest (ROI), focusing on the top part of the bottle
    roi_height = int(height * 0.2)  # Top 20% of the bottle
    roi = image[int(y-height/2):int(y-height/2)+roi_height, int(x-width/2):int(x+width/2)]

    # Convert ROI to HSV
    hsv = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)

    # Define color ranges in HSV
    blue_lower = np.array([100, 100, 100])
    blue_upper = np.array([130, 255, 255])
    orange_lower = np.array([5, 150, 150])
    orange_upper = np.array([25, 255, 255])
    yellow_lower = np.array([20, 100, 100])
    yellow_upper = np.array([35, 255, 255])

    # Create color masks
    blue_mask = cv2.inRange(hsv, blue_lower, blue_upper)
    orange_mask = cv2.inRange(hsv, orange_lower, orange_upper)
    yellow_mask = cv2.inRange(hsv, yellow_lower, yellow_upper)

    # Calculate the percentage of each color
    blue_percent = (blue_mask > 0).mean() * 100
    orange_percent = (orange_mask > 0).mean() * 100
    yellow_percent = (yellow_mask > 0).mean() * 100

    # Determine the dominant color
    colors = {'blue': blue_percent, 'orange': orange_percent, 'yellow': yellow_percent}
    dominant_color = max(colors, key=colors.get)

    # Visualize the HSV image and color masks
    plt.figure(figsize=(15, 5))
    plt.subplot(141), plt.imshow(cv2.cvtColor(roi, cv2.COLOR_BGR2RGB)), plt.title('Original ROI')
    plt.subplot(142), plt.imshow(hsv), plt.title('HSV Image')
    plt.subplot(143), plt.imshow(blue_mask, cmap='gray'), plt.title('Blue Mask')
    plt.subplot(144), plt.imshow(orange_mask, cmap='gray'), plt.title('Orange Mask')
    plt.tight_layout()
    plt.show()

    return dominant_color, colors

Step #4: Integrate Bottle Detection with Color Detection

With both bottle detection and color detection functions ready, we can put them together to analyze each detected bottle’s color.

image = cv2.imread('path/to/image')

for detection in detections:
    x, y, width, height = detection['x'], detection['y'], detection['width'], detection['height']
    dominant_color, colors = detect_color(image, x, y, width, height)
    print(f"Dominant Color: {dominant_color}")
    print(f"Color Percentages: Blue: {colors['blue']:.2f}%, Orange: {colors['orange']:.2f}%, Yellow: {colors['yellow']:.2f}%")

For each detected bottle, the code extracts its position and size and then calls the detect_color function to determine the cap color. The results are printed, showing the dominant color and the percentages of blue, orange, and yellow.

Challenges and Considerations

While using computer vision for color detection has many advantages, it also comes with its own set of challenges that must be considered to achieve accurate results.

One of the biggest challenges is dealing with different lighting conditions. Colors can look different under various lights, making it hard to detect them accurately. The HSV color space helps because it separates the color itself (hue) from how bright or dark it is (value) and how intense the color appears (saturation). However, even with HSV, you often need to fine-tune the settings. Adjusting the ranges for hue, saturation, and value can help make sure that colors are detected consistently, no matter the lighting.

Shadows and reflective surfaces are other common problems. Shadows can make parts of an image darker, which can throw off color detection, while reflections can change how colors appear. These issues can add "noise" to the image. To overcome this, you can use preprocessing techniques like smoothing filters or lighting correction to clean up the image before trying to detect colors. After detecting the colors, post-processing methods, such as morphological operations, can help refine the results and remove any leftover noise.

Advanced denoising techniques can be useful for more complex situations, like when the lighting is constantly changing or reflective surfaces are involved. Tools like NVIDIA Real-Time Denoisers (NRD) are designed to handle these challenges by cleaning up noise from shadows, reflections, and other tricky light effects while keeping important visual details. Techniques like ReBLUR, SIGMA, and ReLAX are also helpful because they provide high-quality results even when the computer's processing power is limited. By using these advanced methods, you can improve the reliability and accuracy of color detection in challenging environments.

Color Sensing Applications and Use Cases

Computer vision for color detection is making a big impact across various industries, driving innovative solutions that boost efficiency and accuracy.

Pharmaceutical Manufacturing 

One interesting application is in the pharmaceutical industry, where color detection helps ensure the quality of products like tablets. For example, Pharma Packaging Systems in England uses computer vision to check the color and size of tablets during production. As the tablets move through the system, the computer analyzes their images to confirm they meet the correct standards. If any tablets don’t match the required color or dimensions, they are automatically removed, and only high-quality products make it to the market.

Robotics

In robotics, color detection plays a crucial role in sorting materials. AMP Robotics, for instance, has developed an AI-driven system that uses computer vision and machine learning to sort different materials quickly and accurately. Their AMP Neuron platform can identify materials based on characteristics like color, texture, shape, and size, and robots can sort detected items at a remarkable speed of 160 pieces per minute. These types of systems are transforming the recycling and waste management industry by making sorting processes faster and more precise.

Agriculture

Computer vision can also help farmers monitor the health of their crops more efficiently. A Brazilian startup called Cromai uses AI-based solutions that analyze the color, shape, and texture of crops to gather important diagnostic insights. Data is gathered from sensors on farming equipment or captured by drones and satellites. With this information, farmers can keep a close eye on their crops, make informed decisions, and ultimately improve their yields.

Conclusion

Using the HSV color space in computer vision provides a strong and accurate way to detect colors in many different areas. By separating color from brightness and intensity, HSV gives more reliable results, especially when the lighting isn't perfect.

Whether it's ensuring high-quality products in the pharmaceutical industry, making sorting faster and more accurate in robotics, or helping farmers monitor their crops, computer vision combined with HSV can create precise and efficient color detection solutions.

Keep Reading

Here are a few related articles about color detection and related areas published on the Roboflow blog:

• Color and product type inspection with computer vision.
• Error proofing in manufacturing with computer vision.
• Computer vision for automotive manufacturing

Cite this Post

Use the following entry to cite this post in your research:

Abirami Vina. (Sep 4, 2024). Color Sensing with Computer Vision. Roboflow Blog: https://blog.roboflow.com/color-sensing-with-computer-vision/

Discuss this Post

If you have any questions about this blog post, start a discussion on the Roboflow Forum.