On a clean, flat image, reading the number on a basketball jersey is trivial. In a live game it is one of the harder problems in sports video. Players turn, jump, and collide, the number bends across the fabric, motion blur smears the digits, and as Marc Zoghby, Co-Founder of PlayVision put it, "a seven can sometimes look like a one." Jersey number recognition is the task of reading those numbers reliably anyway, so a system can tell which player is which.
It matters because the number is how a vision pipeline holds onto a player's identity. Detection finds the bodies and tracking follows them frame to frame, but once players cross, collide, or leave the frame, the tracker needs something stable to re-anchor to. The jersey number is that anchor.
In a recent Roboflow webinar, Marc walked through how his team handles this as one piece of a larger system that turns full game footage into coaching reports. His take on jersey numbers is a useful lesson for anyone building real-world computer vision: the off-the-shelf model gets you started, and then the real work begins.
What Jersey Number Recognition Means
Jersey number recognition is a specialized form of optical character recognition, or OCR, applied to athletes in motion. A general OCR model reads text from documents, signs, and labels. Jersey number recognition reads one or two digits off a curved, moving, often partly hidden surface, then attributes that number to a tracked player.
The task usually sits inside a pipeline. First a detector locates each player on the court. Roboflow's RF-DETR is built for this kind of real-time detection. Then an OCR model reads the number inside each player crop, and the result is tied back to the tracked identity so the system knows that number 23 in this frame is the same number 23 it was following two seconds ago.
Roboflow's guide to detecting and identifying basketball players walks through the full version of that loop.
Why it Matters for Player Tracking
Without reliable numbers, tracking falls apart the moment players interact, which in basketball is constantly. Uniforms on the same team look almost identical, bodies occlude each other under the basket, and the camera pans and zooms. A tracker that loses an identity has to guess, and guesses become errors in every downstream statistic.
For a product like PlayVision, that identity is the foundation of everything coaches actually care about. The bounding boxes are not the point. As Marc described it, coaches do not care that the boxes look cool, they care about the metrics derived from them: passing accuracy, paint touches, contested versus uncontested shots, points per possession.
Every one of those stats depends on knowing which player did what, and that depends on reading the number on the back of the jersey. Get the identity wrong and a guard's passing accuracy is attributed to the wrong player, which is worse than no stat at all.
Why Off-the-Shelf OCR is Not Enough
This is the part Marc is most direct about, and it generalizes well beyond basketball. There are hundreds of OCR models available, and a typical one has no trouble with numbers printed clearly on a flat surface. Sports film is the opposite of that. Players are dynamic, the angle of the numbers changes constantly, and the digits distort until a seven reads like a one. An OCR model that was never trained on those conditions will fail on exactly the frames you need.
The fix is fine-tuning on data that looks like what the model will actually see. That means building a data set of real jersey numbers at game angles, under game lighting, with game motion blur, and training the model on it. PlayVision uses Roboflow to manage that data and to expand it with preprocessing: adding blur, noise, and saturation and hue shifts, and even desaturating to black and white so the model learns the shape of a number rather than latching onto color or a stray round object. This overview of image augmentation covers those techniques.
The data set is not a one-time build. Marc treats it as a living part of the tech stack, the same as code. His team runs their current models on new footage, filters for low-confidence detections and frames where the counts look wrong, and routes those failures back for correction and retraining.
That is active learning, and it is how recognition that started rough becomes reliable across new venues, camera heights, and lighting. Open starting points help too: as Marc noted, you can pull a few hundred open sports datasets to train an early model rather than collecting everything from scratch.
What the Webinar Shows
See the full PlayVision pipeline running on a live game: player detection, pose, ball tracking, 2D court mapping, and event detection, all feeding the coaching report. Marc's closing advice is the kind worth hearing: build something and put it in front of reality fast, because the edge cases you did not anticipate are where the real learning is.
Build Jersey Number Recognition for Sports
Watch the full conversation with Marc Zoghby of PlayVision on building computer vision for basketball analytics.
If you want to build jersey number recognition or the tracking pipeline around it, you can start on Roboflow.
Cite this Post
Use the following entry to cite this post in your research:
Contributing Writer. (May 4, 2026). How to Build Jersey Number Recognition for Sports. Roboflow Blog: https://blog.roboflow.com/jersey-number-recognition-for-sports/