How This Fulbright Scholar is Using Computer Vision to Protect Endangered Species

The below post is by Kasim Rafiq, a conservationist, Fulbright Scholar, and National Geographic Explorer studying at UC Santa Cruz. Kasim holds a PhD in Wildlife Ecology from Liverpool John Moores University.

Introduction

Nature underpins everything. From our global economy to our wellbeing and survival. Everything relies on nature.

Unfortunately, we’re in a biodiversity crisis, with millions of plants and animals at risk of extinction. The problems facing wildlife are complex, but data can help cut through the noise.  Data on wildlife population numbers can help divert our limited resources to the right places, at the right times, similar to doctors treating patients in a pandemic. Ultimately, data helps increase the number of species that can be saved.

Yet here’s the catch. Data is expensive to collect. In nature, we rarely have exact counts of animals, and instead we use various techniques, including camera traps and drones, to make estimates.

The costs of these techniques make monitoring difficult to scale across space and time. Without knowing what’s out there, making conservation decisions based on data becomes impossible.

Using Computer Vision for Wildlife Surveys

That’s where I come in. I’m Kasim, and I’m a wildlife biologist at UC Santa Cruz.  In 2019, I discovered that photographs taken by tourists could be used to get accurate counts of African predators at 97 percent of the cost of other commonly used methods. At the time I said that

‘with a coordinated effort to collect tourist photographs… we could have a real opportunity for continuous, rapid assessment of wildlife populations in high-value tourism areas’

Today, I’m turning to computer vision and edge machine learning to solve the problem of scale and introduce this survey approach to some of Africa’s most iconic landscapes.

The idea is simple. Use vehicle mounted cameras to automatically detect and photograph wildlife on safari game drives. Use that data for species surveys.

This idea was compelling enough to get me a rapid prototyping grant from ConservationX, an organization dedicated to using technology to solve some of the planet’s greatest challenges, and over the last three months building that prototype is exactly what I’ve been doing.

Before we dive into the workflow, a disclaimer. I’m an expert in animal behaviour and field work, particularly vehicle recovery, but not in hardware and machine learning. This means that most of the workflow I’ll outline involved no code and is accessible to anyone with some patience.

Data Collection, Generation, and Labelling

In order to build a system that takes pictures of leopards, you first need to teach it what a leopard looks like. To do that, I had to collect and label enough data to train an algorithm on what makes a leopard a leopard. (Perhaps one of life’s unanswered questions.)

The challenging thing about leopards, though, is that they’re not the easiest cat to find. They like to hang around by themselves, have big home territories, and prefer the thickest, nastiest bushes you can think of.  Luckily, my collaborators and I at Botswana Predator Conservation have spent a lot of time finding them, which means we had access to a rich dataset from researchers, tourists, and camera traps. In all, we had over 1,800 images of 40 leopards in various poses.

These images were annotated by collaborators over at WildMe who we are working with in order to develop computer vision algorithms to tell individual leopards apart from one another for a separate project.

Data Preparation and Augmentation

Once the images were labelled by WildMe, the next challenge was getting it into the right format for training. In the end, I decided to use the Google Cloud AI platform for model training. It was simple, required no coding, and needed no specialist hardware.

The challenge for me was converting the annotated dataset from COCO to the format needed by Google.  This might be simple for anyone with domain expertise, but as an outsider, I spent several hours searching for an answer and was stumped. This is where Roboflow enters the story.

In addition to the features that allow data to be smoothly preprocessed, augmented, and health checked, Roboflow Pro has an export feature that allows you to convert and export your datasets directly to model training platforms like Google Cloud.

Something that had previously taken several unsuccessful hours of Googling took less than ten minutes with Roboflow.

I also took advantage of the augmentation features in Roboflow to test how augmentations that simulate real world scenarios, like camera blur from photographs, might improve the trained models (more on that later).

Modeling

Since these models would need to run on devices, providing real-time inference, I opted to use Google Cloud’s AutoML Vision Edge models.  These are low-latency, high accuracy models, optimized for running on the edge, that can be exported to TensorFlow lite, which made them ideal for fast and easy integration with my hardware stack.

We also tested how Roboflow augmented datasets fared against non-augmented datasets. The answer was fairly well.  With data augmentation, the training set increased by 67 percent and average model precision increased from 0.90 to 0.92 – a modest but nice increase for no extra effort.

Deployment

To deploy, the models were loaded into a Raspberry Pi pan-tilt set up to create a low-fidelity prototype for the ConservationX prize.  Models were then taken on a socially distanced quarantined safari to pick out leopards.

For where we’re going next, we have a number of new milestones. Our next hurdles are to expand the model training to detect a wider range of wildlife and to iterate on the hardware so that we can build something robust enough to survive an adventure in the African bush.

How You Can Help

If you’re a software or hardware developer, start-up, or technology enthusiast and are interested in using technology to support nature conservation, I’d love to hear from you.

To get in touch, or even just to keep track of the project and my other leopard filled work, you can follow me on Twitter or send me an email (kasim.rafiq [at] hotmail.co.uk).

In return, I’m sure you’ll get a sneak peek into some of Africa’s most iconic species and maybe pick up a story or two for your customers.

Contributors

Thanks for this project go out to the various people and organisations involved, including Ankita Shukla, Leigh Johnson, Ed Miller, Gautam Shah, ConservationX, WildMe, Botswana Predator Conservation, and of course, Roboflow.