One of the coolest and most challenging hobbies I've picked up in the last 10 years has been building and maintaining a saltwater reef tank in my home. Most people that take up this hobby start on the easy side; get a large tank (100 gallons or so), plop in a few fish made famous by Pixar, and basically ignore it until you have to do the once-a-month water changes.

Setups like this are really nice because they can be very forgiving when the chemical balance gets off. If a few fish die, or if the pH in the water gets a little off, or if your little niece casually tosses her dripping-with-drool-pacifier into the tank... it's no big deal. Just add a neutralizing chemical, change the water a few times, and everything is back to normal pretty quickly. This is a perfect way to go for all beginners!

Not for me though.

I can't resist taking on a challenging hobby. When I was doing research before investing in this hobby (yes, investing…lots of it), everyone told me don't get anything smaller than 50 gallons.

"Small tanks are way too difficult for a beginner!" - they said.

"Pfft…challenge accepted." - I retorted.

Even though I felt confident about taking on this new challenge, I still wanted to be smart about it since this can be quite an expensive hobby. Especially when you're just starting out. The six-plus-week process of setting up and cycling your tank sets the stage on if everything will survive…or not. And if you do end up killing everything and need to start over, it can be even more expensive. So I started in the middle of the salt water challenge spectrum: a BioCube tank.

The BioCube tanks are considered a "Nano" tank (under 30 gals), and have all the add-in bells and whistles of larger tank built right in (like lights and a filtration system). I figured this might be a good way to fulfill my need for a challenge while having some safeguards in place just in case things go south.

Fortunately for me, all has worked out! In fact, I've been able to successfully spawn (read: breed) clownfish! And it's a pretty big hit with the nieces and nephews. So much so, we usually play a game with it right when they walk into the house; who can find the Sexy Shrimp first?!

Sexy Shrimp are some of the tiniest reef safe shrimp you can keep. And they are fun to watch! Officially called Thor amboinensis, the "Sexy Shrimp" nick name comes from its unusual body movements. When latched on to a coral or just walking around, the Sexy Shrimp sways its abdomen back and forth with exotic flair.

Sexy Shrimp are super fun to watch, but unfortunately they are pretty hard to find. Especially if you are around 5 years old and have a TikTok attention span. So to make it easier for everyone to find these little critters, I created a Sexy Shrimp detector with Roboflow!

Project Steps

Creating a project

Creating a project with Roboflow was super simple:

Name your project

• I chose to call it Detecting and Classifying Thor Amboinensis since I wanted to find them and potentially figure out which were males or females

Choose your project type

• I choose Object Detection since simple detection was the main goal of this project

Create an Annotation Group for labeling

• Surprise, mine was Sexy Shrimp

Uploading images

Beyond finding the Sexy Shrimp in my tank, I was also interested in identifying their sex so I could breed them. The bad news is I only had a few pictures, very little variety, and any time I got too close to the tank, they swam off. To the Googs!

Google had more than enough images (surprise I know) for me to get a model trained, so I grabbed a few dozen to get started!

Editors note: Windows 10 defaults image downloads from Google as .jfif images which is not a current standard. You will need to convert the images to .jpg in order for this to work properly.

Once I felt like I had a variety of male and female Sexy Shrimp images, I used the drag and drop feature to quickly upload all of the images I wanted to use.

Annotating Images

Although this process takes a little bit longer than uploading, it was also very easy to quickly create a bounding box around the shrimp, edit them for the perfect fit, and create subclasses that will help the model identify if the shrimp are male or female!

After I annotated and reviewed my own work, I did a basic Health Check to get a feel for the number of annotations and the balance of classes before moving on to the Preprocessing and Augmentations stages where I can really boost the performance of my model.

While reviewing the Health Check data, I learned that the Class Balance of labels was pretty good, but the Dimension Insights chart was telling me my image size distribution wasn't great. Even though I had a variety of image sizes (small, medium, and jumbo), the median width and height was really, really small; 194px by 261px. This is one of the downfalls of using Google Image Search for my dataset images I guess.

At this point, I could upload a few larger images to balance things out, but I was eager to press on, so with the advice given by our Machine Learning Lead, Jacob Solawetz, I decided to head over to pre-processing and utilize the Resizing step to unify the images sizes so the model doesn't train on super small images of the shrimp.

Preprocessing and Augmentation

My coworker, Jacob, and I worked together to figure out the optimal preprocessing and augmentation steps we should use in order to maximize the success of the model. All of the versions were auto-oriented and resized, but we ended up generating five unique versions for specific reasons:

• The first version had labels remapped to remove the distinction of a shrimp being male or female.
• This was partly due to the complexity of identifying the difference between the male and female; a white stripe verse a split stripe on their tail.
• The second version was also remapped, but we tiled the images.
• Since the objects were pretty small, we wanted to maximize the models performance. Read more about Tackling Small Object Problem in Object Detection.
• The third version had the original labeling and no tiling.
• This version is the closest to what I originally had in mind, so we wanted to test the performance compared to the other models.
• The fourth version had the original labeling with tiling.
• The final version had no changes to labels or any preprocessing / augmentation steps.
• We did this to keep an untouched version.

Train

Now that we created all the preprocessing and augmentation versions we thought were the most valuable, it was time to train the models and see which one might come out on top. Here's the results:

There were some pretty big differences in the mean average precision between the different models, but the first one scored really high. Now it's time to see how it worked in the real world!

Deploy & Use

Since I don't have a lot of technical background and was testing this on my own tank without any edge devices, Jacob and I agreed using the the model in real time with the webcam on my computer was the way to go.

Here's what we got:

Yep the video is short, but in my defense it's kinda cumbersome to capture these shrimp in the back of the tank with my laptop

What's Next

Although we were successful in being able to detect the shrimp in my tank, it also detected a few other things as shrimp it should not have. Next I want to grab a few more images of fish, snails, and other items it detected as shrimp and add them to the dataset to make the model even better!