Making a Handheld Card Counter on the OAK-D-Lite

The portability of the OAK-D-Lite gives us the power to bring computer vision powered solutions anywhere on earth - including your local casino!

At Roboflow, we have been long time supporters of the ecosystem of OAK devices and cannot wait to see what applications get built with the new OAK-D-Lite device.

While we focus on counting cards in blackjack in this example, we want you to see this post as a guide on how you might deploy custom vision models to your OAK-D-Lite in general.

And the best part is that even before your OAK-D-Lite arrives from the kickstarter, you can get started building your model and application today by testing against hosted APIs before you bring your technology to the edge!

A Brief Intro to Card Counting in BlackJack

BlackJack is a popular game played at casinos where the player has a discrete set of decisions to "stay" or "hit", weighing the benefit of improving their hand against the risk of going over 21 and "busting".

Normally, if you play perfect BlackJack - you can only reach a nearly break even point in expected value against the house. That is, probabilistically BlackJack is a losing game.

But if you are able to keep track of cards that have been previously dealt, you can tilt the odds in your favor. Keeping track of a long string cards in your head is exceedingly difficult - so why not have your OAK do it for you?

The Playing Card Dataset On Roboflow Universe

To start, we need a dataset that represents the objects we want our OAK-D-Lite to detect in our production environment. Lucky for us, Ritesh at Augmented Startups has already assembled his playing card dataset for us and shared it on Roboflow Universe.

With this dataset, we will be able to train a computer vision model to detect the numbers and suits of each playing card that falls on the table.

The Playing Card Model on Roboflow Universe

You will notice that on Roboflow Universe, a model has already been trained on the playing cards dataset, with various deployment destinations.

Training on Roboflow optimizes a computer vision model for your dataset and converts it to the deployment destination of your choosing, including the OAK-D-Lite!

Testing Our Card Model on the Hosted API

Before we move down to our OAK device, we should test our model on some example images on the hosted playing cards API. There we can post an example image and see the predicted response.

Example Image:

Example Response:

Example Response JSON:

{
    "predictions": [
        {
            "x": 1093,
            "y": 908,
            "width": 74,
            "height": 168,
            "class": "QH",
            "confidence": 0.885
        },
        {
            "x": 822,
            "y": 303.5,
            "width": 74,
            "height": 163,
            "class": "QH",
            "confidence": 0.884
        },
        {
            "x": 299,
            "y": 296,
            "width": 74,
            "height": 172,
            "class": "KH",
            "confidence": 0.873
        },
        {
            "x": 734,
            "y": 822.5,
            "width": 62,
            "height": 131,
            "class": "KH",
            "confidence": 0.66
        }
    ]
}

Deploy Our Card Model to the OAK-D-Lite

Until now, our model has been running on a cloud server. To take embed this model in our device on the OAK-D-Lite, we need to convert it to run on the MyriadX and write pre and post processing routines. Luckily Roboflow takes care of these steps for us!

To start our inference server on the OAK-D-Lite we need only run the following commands on our host device after our model has been trained on Roboflow:

Stand up OAK Server:

sudo docker run --rm \
    --privileged \
    -v /dev/bus/usb:/dev/bus/usb \
    --device-cgroup-rule='c 189:* rmw' \
    -e DISPLAY=$DISPLAY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -p 9001:9001 \
    roboflow/oak-inference-server:latest

Infer from OAK Server - GET:

http://localhost:9001/[YOUR_ENDPOINT]?access_token=[YOUR_ACCESS_TOKEN]

Response from your OAK-D-Lite at realtime speeds:

{
    "predictions": [
        {
            "x": 1093,
            "y": 908,
            "width": 74,
            "height": 168,
            "class": "QH",
            "confidence": 0.885
        },
        {
            "x": 822,
            "y": 303.5,
            "width": 74,
            "height": 163,
            "class": "QH",
            "confidence": 0.884
        },
        {
            "x": 299,
            "y": 296,
            "width": 74,
            "height": 172,
            "class": "KH",
            "confidence": 0.873
        },
        {
            "x": 734,
            "y": 822.5,
            "width": 62,
            "height": 131,
            "class": "KH",
            "confidence": 0.66
        }
    ]
}

More OAK Server Docs.

Hitting the Casinos

Now that we have our card model inferring on the OAK-D-Lite, we need to start writing logic to count cards and recommend strategies, a fun adventure in implementation and development.

Check out this video where Roboflow CTO Brad Dwyer does as much in a couple of hours in a live YouTube hackathon. From what I've heard, it's been working quite well for him at the local casinos 🤑.

Conclusion

We hope you found this as fun as we did and we can't wait to see what custom models and solutions you build on your OAK-D-Lite. Happy training, and most certainly, happy deploying!