Using Computer Vision to Win at Duck Hunt

This post is a guest post written by Abhinav Mandava, who is a Master of Applied Computer Science student at Dalhousie University. Abhinav worked as an Associate Software Engineer at UnitedHealth Group before pursuing his Master's degree. He is curious, is passionate about technology, and loves to get hands-on experience with the latest trends in artificial intelligence, IoT, big data, and cloud-native application development.

The video game Duck Hunt is one many may recognize from arcades or older game systems – it's been around since the 1980s! If you're not familiar, Duck Hunt is a video game in which ducks appear on screen and you have to use your controller to aim a crosshair over the moving ducks and shoot them.

Here, I'm leveraging object detection models and Roboflow to create an aimbot (which automates aiming and firing for the player) for Duck Hunt.

Note: One has to keep in mind that this is just a fun project built while exploring and experimenting with computer vision. Unauthorized use of aimbots is forbidden in gaming and they spoil users’ experience of the game!

Can we use object detection to automate identifying moving objects on a screen?

I gathered 253 screenshots of the gameplay which included both flying ducks and falling (dead) ducks. I annotated the images with Roboflow and was able to easily export the data in the format required by the models I chose. The user experience for preprocessing is amazing. From start to finish, it took me less than 20 minutes to prepare and export my dataset.

I used both the EfficientDet-D0 model and the TensorFlow Object Detection API. After fitting a few models, Roboflow's Dataset Health Check helped me understand why my models initially performed poorly – the crosshair was sometimes incorrectly identified as a black duck. I gathered and annotated more images of the black ducks, which improved my model.

With advancements in computer vision, we have many state-of-the-art models for object recognition, object detection, object tracking, pose estimation, content-based image retrieval, motion analysis, and more. I found Roboflow’s Model Library to be the best, most organized resource for finding and playing around with state-of-the art computer vision models.

This also isn't limited to gameplay – automating the identification of moving objects on a screen can be really flexible. For example, being able to find a mouse or cursor on a computer screen might allow you to automate some of your regular work.

I look forward to building more computer vision based projects and Roboflow would be my go-to platform without a second thought!