How to Deploy Computer Vision Models to Jetson Orin Nano
Paul Guerrie
Published Jul 24, 2023 • 7 min read

NVIDIA's newest Jetson Orin line promises a leap forward in computational efficiency and compatibility. Running on JetPack 5.0 and beyond, these compact yet powerful devices are akin to Raspberry Pis, but with the considerable addition of an integrated GPU.

Nvidia Jetson Orin Family

The Jetson Orin Nano, in particular, has captured the attention of many tech enthusiasts and professionals alike.

Today, we will explore how to use Roboflow to run a computer vision model on a Jetson Orin Nano. We will set up the device to run a rail yard instance segmentation model capable of detecting trains, rail tracks, people, cars, and traffic lights. Trained using nearly 3,000 images, this model could be used for a variety of applications in any railway setting.

Try the model on Roboflow Universe

At the end of this step-by-step tutorial, you'll have an instance segmentation model running on a Jetson Orin Nano using the Roboflow Inference Server.

Let's get started!

Selecting the right camera and lens is foundational to the success of any computer vision project. Download the “Choosing Cameras and Lenses for Computer Vision” report.

The Roboflow Inference Server

Roboflow is a versatile platform that allows users to upload data, annotate data, train models, upload pre-trained models, and deploy to 10 different targets across multiple vision task types. Roboflow is designed to streamline the process of developing and implementing computer vision solutions. Once you have a trained model on Roboflow, you're all set to perform inferences.

For this tutorial, we will use the Roboflow Inference Server. The Inference Server is designed to simplify the process of running inference on almost any device. Even better, Roboflow publishes Docker images that are specifically designed for Jetsons. These images leverage TensorRT (TRT) execution, ensuring that you get the best possible performance on your Jetson Orin Nano.

How to Use Computer Vision Models on the Jetson Orin Nano

Step #1: Setup

To get started, first ensure that your Jetson Orin Nano is up and running. Once you have set up your device, you can start the Roboflow Inference Server using a simple Docker run command.

docker run -it --rm --network=host --runtime=nvidia roboflow/roboflow-inference-server-trt-jetson-5.1.1

Step #2: Client

With the server running, the next step is to access your trained model. Get your model ID, model version, and API key from Roboflow. These will be required to perform inference. Since the train detection model we will be using is publicly available on Roboflow Universe, you will be able to use it with your own API key, even though it belongs to another user.

api_key=<YOUR_API_KEY>
model="train-detection-4ud7c"
version=1

image_url = "https://source.roboflow.com/zD7y6XOoQnh7WC160Ae7/yA6pCzno5RW5tc3LjgSR/original.jpg"
image = Image.open(requests.get(image_url, stream=True).raw) #Reading the image from a URL for demonstration purposes
buffered = BytesIO()
image.save(buffered, quality=100, format="JPEG")
img_str = base64.b64encode(buffered.getvalue())
img_str = img_str.decode("ascii")

Now, you're ready to create a POST request with your image. This will send the image to the Roboflow Inference Server, which will process the image using your trained model. The server will then return a response containing the inference results. These results can be parsed to extract meaningful information about the objects detected in the image.

headers = {"Content-Type": "application/x-www-form-urlencoded"}
res = requests.post(
    f"http://localhost:9001/{model}/{version}?api_key={api_key}",
    data=img_str,
    headers=headers,
)
Visualized Instance Segmentation Detections

Response JSON >>

{'time': 0.38400292200003605, 'image': {'width': 640, 'height': 640}, 'predictions': [{'x': 344.5, 'y': 356.5, 'width': 447.0, 'height': 285.0, 'confidence': 0.86333829164505, 'class': 'trains', 'points': [{'x': 211.0, 'y': 496.02762430973013}, {'x': 189.0, 'y': 495.0035224759666}, {'x': 167.0, 'y': 489.1387147978161}, {'x': 157.0, 'y': 490.01071429139967}, {'x': 147.0, 'y': 489.0421529125008}, {'x': 143.0, 'y': 487.00776708882273}, {'x': 136.99892543000965, 'y': 479.0}, {'x': 134.9863251136244, 'y': 465.0}, {'x': 135.8435557336554, 'y': 446.0}, {'x': 132.0, 'y': 442.1472471217215}, {'x': 121.93135174165441, 'y': 441.0}, {'x': 121.71417511453723, 'y': 405.0}, {'x': 124.99055916342549, 'y': 390.0}, {'x': 121.98113772899411, 'y': 327.0}, {'x': 131.72436346079536, 'y': 292.0}, {'x': 131.9699544312003, 'y': 272.0}, {'x': 134.80248522292078, 'y': 265.0}, {'x': 135.89825541757145, 'y': 256.0}, {'x': 139.97661368688298, 'y': 249.0}, {'x': 154.0, 'y': 235.91602737094632}, {'x': 163.8276876149538, 'y': 230.0}, {'x': 163.93430978612648, 'y': 222.0}, {'x': 170.0, 'y': 215.99964936178534}, {'x': 182.0, 'y': 214.54769262907328}, {'x': 251.0, 'y': 214.9366826546555}, {'x': 258.0, 'y': 215.99296504404873}, {'x': 275.0, 'y': 222.7898217831606}, {'x': 314.0, 'y': 224.95071971025897}, {'x': 323.0, 'y': 227.72684228214152}, {'x': 339.0, 'y': 227.99914156762952}, {'x': 350.0, 'y': 231.75960427569288}, {'x': 387.0, 'y': 235.9695636559599}, {'x': 397.0, 'y': 239.80995738975906}, {'x': 405.0, 'y': 240.95414278784526}, {'x': 410.0, 'y': 243.77225933212767}, {'x': 422.0, 'y': 244.97756419187425}, {'x': 427.0, 'y': 247.7944311651627}, {'x': 434.0, 'y': 248.9578929121521}, {'x': 438.0, 'y': 251.81909494531223}, {'x': 465.0, 'y': 255.76417338985402}, {'x': 488.0, 'y': 256.99589707880045}, {'x': 511.0, 'y': 260.9960116174157}, {'x': 543.0, 'y': 270.8007421193313}, {'x': 553.0, 'y': 271.9695867014859}, {'x': 560.0, 'y': 275.97180403485805}, {'x': 566.1069342230705, 'y': 282.0}, {'x': 567.423402379246, 'y': 293.0}, {'x': 567.3368111094487, 'y': 368.0}, {'x': 567.1878779016541, 'y': 370.0}, {'x': 562.0, 'y': 374.03374723482784}, {'x': 526.0, 'y': 384.17174116560324}, {'x': 490.0, 'y': 402.01212950315556}, {'x': 463.0, 'y': 411.0200224866675}, {'x': 453.0, 'y': 412.1814660607442}, {'x': 435.0, 'y': 419.0492628480348}, {'x': 420.0, 'y': 422.1367654091879}, {'x': 395.0, 'y': 434.0247495072242}, {'x': 360.0, 'y': 444.18690960854866}, {'x': 342.0, 'y': 452.13833433539025}, {'x': 331.0, 'y': 459.0174171550626}, {'x': 299.0, 'y': 468.1290351020149}, {'x': 290.0, 'y': 474.06291648539934}, {'x': 270.0, 'y': 477.11673686565933}, {'x': 262.0, 'y': 480.10187678452974}, {'x': 242.0, 'y': 491.0704799272543}, {'x': 220.0, 'y': 493.1534354804707}, {'x': 211.0, 'y': 496.02762430973013}]}, {'x': 573.5, 'y': 547.0, 'width': 129.0, 'height': 184.0, 'confidence': 0.4539775848388672, 'class': 'rail-track', 'points': [{'x': 638.0, 'y': 638.4867329378259}, {'x': 508.71026575272236, 'y': 638.0}, {'x': 560.9724040683512, 'y': 561.0}, {'x': 582.7922206761691, 'y': 534.0}, {'x': 600.9703341207595, 'y': 505.0}, {'x': 637.0, 'y': 455.70957520864266}, {'x': 638.2904247913574, 'y': 456.0}, {'x': 638.4583694666649, 'y': 462.0}, {'x': 638.0, 'y': 638.4867329378259}]}, {'x': 115.0, 'y': 538.0, 'width': 200.0, 'height': 92.0, 'confidence': 0.12480044364929199, 'class': 'rail-track', 'points': [{'x': 18.0, 'y': 573.030377477556}, {'x': 15.892087883526742, 'y': 573.0}, {'x': 15.50173146105391, 'y': 553.0}, {'x': 15.78406635851133, 'y': 535.0}, {'x': 18.0, 'y': 532.9647023933609}, {'x': 26.0, 'y': 531.7793789742841}, {'x': 56.0, 'y': 520.9479546502524}, {'x': 69.0, 'y': 518.8065642996469}, {'x': 86.0, 'y': 511.95414662787044}, {'x': 94.0, 'y': 510.8113719207422}, {'x': 110.0, 'y': 503.94069896245253}, {'x': 130.0, 'y': 498.8217109173472}, {'x': 135.0, 'y': 495.99189131609484}, {'x': 149.0, 'y': 494.83271551287817}, {'x': 153.0, 'y': 492.9683112488088}, {'x': 168.0, 'y': 492.9731195053897}, {'x': 180.0, 'y': 496.90660100221163}, {'x': 199.0, 'y': 495.9837238845184}, {'x': 211.00977528358203, 'y': 501.0}, {'x': 211.01303966904248, 'y': 504.0}, {'x': 209.0, 'y': 506.0218221371695}, {'x': 204.0, 'y': 507.09926858657803}, {'x': 194.0, 'y': 514.0627118307981}, {'x': 141.0, 'y': 531.1905663065942}, {'x': 129.0, 'y': 537.0211394333895}, {'x': 116.0, 'y': 540.1746201167995}, {'x': 107.0, 'y': 545.0249450486224}, {'x': 93.0, 'y': 548.1377122582276}, {'x': 82.0, 'y': 555.0361537703855}, {'x': 70.0, 'y': 558.1210621599129}, {'x': 65.0, 'y': 561.0483066177301}, {'x': 40.0, 'y': 565.1131887877335}, {'x': 18.0, 'y': 573.030377477556}]}, {'x': 617.5, 'y': 310.5, 'width': 43.0, 'height': 37.0, 'confidence': 0.051128894090652466, 'class': 'trains', 'points': [{'x': 638.0, 'y': 329.28539285772223}, {'x': 633.0, 'y': 329.0144991965105}, {'x': 629.0, 'y': 326.0838700633589}, {'x': 621.0, 'y': 326.0214567735048}, {'x': 618.0, 'y': 324.13700834965846}, {'x': 612.0, 'y': 324.02386377850274}, {'x': 597.0, 'y': 320.03883596628896}, {'x': 595.6757821153547, 'y': 315.0}, {'x': 595.9584850892219, 'y': 292.0}, {'x': 638.0, 'y': 291.6239770224582}, {'x': 638.0, 'y': 329.28539285772223}]}, {'x': 27.0, 'y': 200.5, 'width': 54.0, 'height': 299.0, 'confidence': 0.018806636333465576, 'class': 'trains', 'points': [{'x': 50.0, 'y': 350.0678690503257}, {'x': 45.0, 'y': 347.04550892354536}, {'x': 41.0, 'y': 350.05961446642635}, {'x': 38.0, 'y': 350.3683066834566}, {'x': 0.8517725391282533, 'y': 350.0}, {'x': 0.9747866838975832, 'y': 348.0}, {'x': 4.940099350538692, 'y': 345.0}, {'x': 3.9927765883490127, 'y': 341.0}, {'x': 7.0, 'y': 332.9791980053038}, {'x': 12.0, 'y': 330.99662069360113}, {'x': 30.0, 'y': 329.9289486479113}, {'x': 32.99649621501977, 'y': 326.0}, {'x': 29.0, 'y': 324.0352620917415}, {'x': 0.9492579827519556, 'y': 322.0}, {'x': 1.0, 'y': 51.736213595698494}, {'x': 17.0, 'y': 52.98177096188778}, {'x': 21.0, 'y': 55.86397854474082}, {'x': 31.0, 'y': 59.832902878704665}, {'x': 39.0, 'y': 60.994526725356494}, {'x': 54.10284436110869, 'y': 68.0}, {'x': 54.046665541361946, 'y': 193.0}, {'x': 52.12380228692915, 'y': 203.0}, {'x': 51.10283931955072, 'y': 233.0}, {'x': 54.01215736711536, 'y': 257.0}, {'x': 54.06960009285053, 'y': 314.0}, {'x': 50.00975058378319, 'y': 322.0}, {'x': 54.019254253358504, 'y': 327.0}, {'x': 54.00828121208217, 'y': 334.0}, {'x': 53.01847237239503, 'y': 345.0}, {'x': 50.0, 'y': 350.0678690503257}]}, {'x': 220.5, 'y': 159.5, 'width': 17.0, 'height': 29.0, 'confidence': 0.016441255807876587, 'class': 'traffic-light', 'points': [{'x': 226.0, 'y': 171.16769038865795}, {'x': 218.0, 'y': 171.12669863641383}, {'x': 211.88245329010874, 'y': 165.0}, {'x': 212.0, 'y': 145.60389001002468}, {'x': 229.3406804695391, 'y': 146.0}, {'x': 229.2320336475416, 'y': 167.0}, {'x': 226.0, 'y': 171.16769038865795}]}]}

Roboflow and Jetson Orin Nano

The combination of the Jetson Orin Nano and Roboflow opens up use cases for computer vision at the edge. Whether you're building a home security system, a wildlife monitoring device, an industrial inspection tool, or a train detection solution, this robust duo has you covered.

Moreover, the small footprint and low power consumption of the Jetson Orin Nano means that you can run computer vision models in almost any environment. Whether you're at the edge or in a constrained setting, deploying a powerful computer vision system is just a few steps away.

Cite this Post

Use the following entry to cite this post in your research:

Paul Guerrie. (Jul 24, 2023). How to Deploy Computer Vision Models to Jetson Orin Nano. Roboflow Blog: https://blog.roboflow.com/how-to-deploy-to-jetson-orin-nano-computer-vision/

Stay Connected
Get the Latest in Computer Vision First
Unsubscribe at any time. Review our Privacy Policy.

Written by

Paul Guerrie
Machine Learning Engineer at Roboflow - Building tools to democratize computer vision.
View more posts

Topics