How to Use Scale-Invariant Feature Transform (SIFT)

Invented in 1999 by David Lowe, Scale-Invariant Feature Transform (SIFT) is a computer vision algorithm for identifying and matching features in an image.

Using SIFT, you can calculate key points that can be matched across two images. If more than a certain number of key points are close to each other in both images, it means that it is likely that one image is a subset of another.

In this guide, we are going to show how to use SIFT in a computer vision pipeline to check for the presence of an object.

Here is the Workflow we will build, which you can try by uploading two images: one with the feature you want to identify (i.e. a product label), and another that you want to check contains the feature (i.e. a full product package).

Without further ado, let’s get started!

SIFT in Computer Vision Systems

There are four steps to apply SIFT in a computer vision application:

1. Calculate key points in an image that contains the feature you want to identify;
2. Calculate key points in an image that you want to check contains a feature, then;
3. Compare the key points and evaluate their distance.
4. Set a threshold of similar points above which it is said that one image contains the other image.

If enough key points are sufficiently close in the two provided images that have been run through SIFT, it can be determined that a feature is in an image.

Let’s walk through all the steps to use SIFT in Roboflow Workflows.

Step #1: Create a Workflow

and click the Workflows tab in the left sidebar. This will take you to the Workflows homepage, from which you can create a workflow.

Click “Create Workflow” to create a Workflow.

A window will appear in which you can choose from several templates. For this guide, select “Custom Workflow”:

Click “Create Workflow” to create your Workflow.

You will be taken into the Workflows editor in which you can configure your Workflow:

With a blank Workflow ready, we can start using the SIFT features in Workflows.

Step #2: Add SIFT Blocks

First, click “Input Image” on the Input block to add an input image to your Workflow:

Set one of the images with the name image and the other with the name feature. feature will be the image that we want to match to image.

Next, click “Add Block” in the top right corner of the Workflows editor, then add two SIFT blocks:

Connect the first SIFT block to take image as the input:

Connect the second SIFT block to take feature as the input:

These two blocks will compute SIFT key points for our input and feature images, respectively.

Step #3: Add SIFT Comparison Block

Now that our Workflow can calculate SIFT points, we can use the SIFT Comparison block to compare the key points between our two SIFT blocks.

Add a SIFT Comparison block:

Configure the block to take in input from the `descriptor` values in both of the SIFT blocks. By default, your SIFT blocks will be called sift and sift_1.

Your configured comparison block should look like this:

At this stage, your Workflow should have three blocks:

1. A SIFT block that accepts an input image;
2. A SIFT block that accepts an image with features you want to find in the input image, and;
3. A SIFT comparison block.

There is one more step to build our Workflow: set a keypoint threshold above which we can say that the input image contains the feature image.

Step #4: Set a SIFT Keypoint Threshold

Click “Add Block” and add an Expression block to your Workflow:

The Expression block allows you to write logic that returns a PASS / FAIL value depending on whether a condition is or is not met.

Click matches in the configuration box to open a window in which you can configure your SIFT workflow.

Create a parameter called matches that references $steps.sift_comparison.good_matches_count:

Then, set a case statement that checks whether the value of matches is greater than a given reference value. This number will vary depending on your use case. For this guide, we have set 100 as a baseline threshold. In general, the bigger the image you use as a template, and the more unique points it has, the higher this number should be. 

Here is what the final configuration looks like:

Click “Save” to save the block configuration.

Finally, click on the Response block and add the Expression output value to the block:

Step #5: Test the Workflow

We are now ready to test our Workflow!

Our Workflow does the following:

1. Accepts two images: a feature image and an input image.
2. Applies SIFT to both images.
3. Calculates the number of similar key points in both images.
4. If the number of similar key points is greater than 100, the system returns PASS.
5. Otherwise, the system returns FAIL.

Here is what the final Workflow looks like:

Click “Run Preview” to test your Workflow.

Then, upload the two images to which you want to apply SIFT.

For this guide, we can use an image of a coffee bag, and an image of the expected label. We can use this to determine if the top of the label has been printed on the bag.

Here are the images:

Once you have uploaded your images, click “Run Preview” again to test the Workflow.

Here is the result from our Workflow:

Our system returned PASS. This is because more than 100 key points matched.

We successfully identified that one image, the label, was in the other image, the coffee bag.

Deploy Your Workflow

You can deploy a Workflow in three ways:

1. To the Roboflow cloud using the Roboflow API;
2. On a Dedicated Deployment server hosted by Roboflow and provisioned exclusively for your use, or;
3. On your own hardware.

The Workflows deployment documentation walks through exactly how to deploy Workflows using the various methods above.

Deploying to the cloud is ideal if you need an API to run your Workflows without having to manage your own hardware. With that said, for use cases where reducing latency is critical, we recommend deploying on your own hardware.

If you deploy your model in the Roboflow cloud, you can run inference on images. If you deploy on a Dedicated Deployment or your own hardware, you can run inference on images, videos, webcam feeds, and RTSP streams.

To deploy a Workflow, click “Deploy Workflow” on any Workflow in your Roboflow Workspace. A window will then open with information about how you can deploy your Workflow.

Conclusion

SIFT is a computer vision algorithm for feature matching. You can use SIFT to identify whether key points from a feature image match a bigger image. This functionality can be used to build a system that checks for the presence of an object in an image. If a certain number of SIFT key points overlap, it can be inferred that the feature image is within the bigger image.

In this guide, we walked through how to use Roboflow Workflows to apply the SIFT algorithm. We created a new Workflow, added two SIFT blocks, then used the SIFT Comparison block to compare key points between both SIFT results. We then used the Expression block to return a PASS / FAIL value depending on whether enough SIFT keypoints matched in both images.

To learn more about applications you can build with Roboflow Workflows, refer to the Workflows Template gallery. To explore more blocks available in Workflows, refer to the Workflow Blocks gallery.