NVIDIA's newest Jetson Orin line promises a leap forward in computational efficiency and compatibility. Running on JetPack 5.0 and beyond, these compact yet powerful devices are akin to Raspberry Pis, but with the considerable addition of an integrated GPU.
The Jetson Orin Nano, in particular, has captured the attention of many tech enthusiasts and professionals alike.
Today, we will explore how to use Roboflow to run a computer vision model on a Jetson Orin Nano. We will set up the device to run a rail yard instance segmentation model capable of detecting trains, rail tracks, people, cars, and traffic lights. Trained using nearly 3,000 images, this model could be used for a variety of applications in any railway setting.
At the end of this step-by-step tutorial, you'll have an instance segmentation model running on a Jetson Orin Nano using the Roboflow Inference Server.
Let's get started!
The Roboflow Inference Server
Roboflow is a versatile platform that allows users to upload data, annotate data, train models, upload pre-trained models, and deploy to 10 different targets across multiple vision task types. Roboflow is designed to streamline the process of developing and implementing computer vision solutions. Once you have a trained model on Roboflow, you're all set to perform inferences.
For this tutorial, we will use the Roboflow Inference Server. The Inference Server is designed to simplify the process of running inference on almost any device. Even better, Roboflow publishes Docker images that are specifically designed for Jetsons. These images leverage TensorRT (TRT) execution, ensuring that you get the best possible performance on your Jetson Orin Nano.
How to Use Computer Vision Models on the Jetson Orin Nano
Step #1: Setup
To get started, first ensure that your Jetson Orin Nano is up and running. Once you have set up your device, you can start the Roboflow Inference Server using a simple Docker run command.
docker run -it --rm --network=host --runtime=nvidia roboflow/roboflow-inference-server-trt-jetson-5.1.1
Step #2: Client
With the server running, the next step is to access your trained model. Get your model ID, model version, and API key from Roboflow. These will be required to perform inference. Since the train detection model we will be using is publicly available on Roboflow Universe, you will be able to use it with your own API key, even though it belongs to another user.
api_key=<YOUR_API_KEY>
model="train-detection-4ud7c"
version=1
image_url = "https://source.roboflow.com/zD7y6XOoQnh7WC160Ae7/yA6pCzno5RW5tc3LjgSR/original.jpg"
image = Image.open(requests.get(image_url, stream=True).raw) #Reading the image from a URL for demonstration purposes
buffered = BytesIO()
image.save(buffered, quality=100, format="JPEG")
img_str = base64.b64encode(buffered.getvalue())
img_str = img_str.decode("ascii")Now, you're ready to create a POST request with your image. This will send the image to the Roboflow Inference Server, which will process the image using your trained model. The server will then return a response containing the inference results. These results can be parsed to extract meaningful information about the objects detected in the image.
headers = {"Content-Type": "application/x-www-form-urlencoded"}
res = requests.post(
f"http://localhost:9001/{model}/{version}?api_key={api_key}",
data=img_str,
headers=headers,
)
Response JSON >>
{'time': 0.38400292200003605, 'image': {'width': 640, 'height': 640}, 'predictions': [{'x': 344.5, 'y': 356.5, 'width': 447.0, 'height': 285.0, 'confidence': 0.86333829164505, 'class': 'trains', 'points': [{'x': 211.0, 'y': 496.02762430973013}, {'x': 189.0, 'y': 495.0035224759666}, {'x': 167.0, 'y': 489.1387147978161}, {'x': 157.0, 'y': 490.01071429139967}, {'x': 147.0, 'y': 489.0421529125008}, {'x': 143.0, 'y': 487.00776708882273}, {'x': 136.99892543000965, 'y': 479.0}, {'x': 134.9863251136244, 'y': 465.0}, {'x': 135.8435557336554, 'y': 446.0}, {'x': 132.0, 'y': 442.1472471217215}, {'x': 121.93135174165441, 'y': 441.0}, {'x': 121.71417511453723, 'y': 405.0}, {'x': 124.99055916342549, 'y': 390.0}, {'x': 121.98113772899411, 'y': 327.0}, {'x': 131.72436346079536, 'y': 292.0}, {'x': 131.9699544312003, 'y': 272.0}, {'x': 134.80248522292078, 'y': 265.0}, {'x': 135.89825541757145, 'y': 256.0}, {'x': 139.97661368688298, 'y': 249.0}, {'x': 154.0, 'y': 235.91602737094632}, {'x': 163.8276876149538, 'y': 230.0}, {'x': 163.93430978612648, 'y': 222.0}, {'x': 170.0, 'y': 215.99964936178534}, {'x': 182.0, 'y': 214.54769262907328}, {'x': 251.0, 'y': 214.9366826546555}, {'x': 258.0, 'y': 215.99296504404873}, {'x': 275.0, 'y': 222.7898217831606}, {'x': 314.0, 'y': 224.95071971025897}, {'x': 323.0, 'y': 227.72684228214152}, {'x': 339.0, 'y': 227.99914156762952}, {'x': 350.0, 'y': 231.75960427569288}, {'x': 387.0, 'y': 235.9695636559599}, {'x': 397.0, 'y': 239.80995738975906}, {'x': 405.0, 'y': 240.95414278784526}, {'x': 410.0, 'y': 243.77225933212767}, {'x': 422.0, 'y': 244.97756419187425}, {'x': 427.0, 'y': 247.7944311651627}, {'x': 434.0, 'y': 248.9578929121521}, {'x': 438.0, 'y': 251.81909494531223}, {'x': 465.0, 'y': 255.76417338985402}, {'x': 488.0, 'y': 256.99589707880045}, {'x': 511.0, 'y': 260.9960116174157}, {'x': 543.0, 'y': 270.8007421193313}, {'x': 553.0, 'y': 271.9695867014859}, {'x': 560.0, 'y': 275.97180403485805}, {'x': 566.1069342230705, 'y': 282.0}, {'x': 567.423402379246, 'y': 293.0}, {'x': 567.3368111094487, 'y': 368.0}, {'x': 567.1878779016541, 'y': 370.0}, {'x': 562.0, 'y': 374.03374723482784}, {'x': 526.0, 'y': 384.17174116560324}, {'x': 490.0, 'y': 402.01212950315556}, {'x': 463.0, 'y': 411.0200224866675}, {'x': 453.0, 'y': 412.1814660607442}, {'x': 435.0, 'y': 419.0492628480348}, {'x': 420.0, 'y': 422.1367654091879}, {'x': 395.0, 'y': 434.0247495072242}, {'x': 360.0, 'y': 444.18690960854866}, {'x': 342.0, 'y': 452.13833433539025}, {'x': 331.0, 'y': 459.0174171550626}, {'x': 299.0, 'y': 468.1290351020149}, {'x': 290.0, 'y': 474.06291648539934}, {'x': 270.0, 'y': 477.11673686565933}, {'x': 262.0, 'y': 480.10187678452974}, {'x': 242.0, 'y': 491.0704799272543}, {'x': 220.0, 'y': 493.1534354804707}, {'x': 211.0, 'y': 496.02762430973013}]}, {'x': 573.5, 'y': 547.0, 'width': 129.0, 'height': 184.0, 'confidence': 0.4539775848388672, 'class': 'rail-track', 'points': [{'x': 638.0, 'y': 638.4867329378259}, {'x': 508.71026575272236, 'y': 638.0}, {'x': 560.9724040683512, 'y': 561.0}, {'x': 582.7922206761691, 'y': 534.0}, {'x': 600.9703341207595, 'y': 505.0}, {'x': 637.0, 'y': 455.70957520864266}, {'x': 638.2904247913574, 'y': 456.0}, {'x': 638.4583694666649, 'y': 462.0}, {'x': 638.0, 'y': 638.4867329378259}]}, {'x': 115.0, 'y': 538.0, 'width': 200.0, 'height': 92.0, 'confidence': 0.12480044364929199, 'class': 'rail-track', 'points': [{'x': 18.0, 'y': 573.030377477556}, {'x': 15.892087883526742, 'y': 573.0}, {'x': 15.50173146105391, 'y': 553.0}, {'x': 15.78406635851133, 'y': 535.0}, {'x': 18.0, 'y': 532.9647023933609}, {'x': 26.0, 'y': 531.7793789742841}, {'x': 56.0, 'y': 520.9479546502524}, {'x': 69.0, 'y': 518.8065642996469}, {'x': 86.0, 'y': 511.95414662787044}, {'x': 94.0, 'y': 510.8113719207422}, {'x': 110.0, 'y': 503.94069896245253}, {'x': 130.0, 'y': 498.8217109173472}, {'x': 135.0, 'y': 495.99189131609484}, {'x': 149.0, 'y': 494.83271551287817}, {'x': 153.0, 'y': 492.9683112488088}, {'x': 168.0, 'y': 492.9731195053897}, {'x': 180.0, 'y': 496.90660100221163}, {'x': 199.0, 'y': 495.9837238845184}, {'x': 211.00977528358203, 'y': 501.0}, {'x': 211.01303966904248, 'y': 504.0}, {'x': 209.0, 'y': 506.0218221371695}, {'x': 204.0, 'y': 507.09926858657803}, {'x': 194.0, 'y': 514.0627118307981}, {'x': 141.0, 'y': 531.1905663065942}, {'x': 129.0, 'y': 537.0211394333895}, {'x': 116.0, 'y': 540.1746201167995}, {'x': 107.0, 'y': 545.0249450486224}, {'x': 93.0, 'y': 548.1377122582276}, {'x': 82.0, 'y': 555.0361537703855}, {'x': 70.0, 'y': 558.1210621599129}, {'x': 65.0, 'y': 561.0483066177301}, {'x': 40.0, 'y': 565.1131887877335}, {'x': 18.0, 'y': 573.030377477556}]}, {'x': 617.5, 'y': 310.5, 'width': 43.0, 'height': 37.0, 'confidence': 0.051128894090652466, 'class': 'trains', 'points': [{'x': 638.0, 'y': 329.28539285772223}, {'x': 633.0, 'y': 329.0144991965105}, {'x': 629.0, 'y': 326.0838700633589}, {'x': 621.0, 'y': 326.0214567735048}, {'x': 618.0, 'y': 324.13700834965846}, {'x': 612.0, 'y': 324.02386377850274}, {'x': 597.0, 'y': 320.03883596628896}, {'x': 595.6757821153547, 'y': 315.0}, {'x': 595.9584850892219, 'y': 292.0}, {'x': 638.0, 'y': 291.6239770224582}, {'x': 638.0, 'y': 329.28539285772223}]}, {'x': 27.0, 'y': 200.5, 'width': 54.0, 'height': 299.0, 'confidence': 0.018806636333465576, 'class': 'trains', 'points': [{'x': 50.0, 'y': 350.0678690503257}, {'x': 45.0, 'y': 347.04550892354536}, {'x': 41.0, 'y': 350.05961446642635}, {'x': 38.0, 'y': 350.3683066834566}, {'x': 0.8517725391282533, 'y': 350.0}, {'x': 0.9747866838975832, 'y': 348.0}, {'x': 4.940099350538692, 'y': 345.0}, {'x': 3.9927765883490127, 'y': 341.0}, {'x': 7.0, 'y': 332.9791980053038}, {'x': 12.0, 'y': 330.99662069360113}, {'x': 30.0, 'y': 329.9289486479113}, {'x': 32.99649621501977, 'y': 326.0}, {'x': 29.0, 'y': 324.0352620917415}, {'x': 0.9492579827519556, 'y': 322.0}, {'x': 1.0, 'y': 51.736213595698494}, {'x': 17.0, 'y': 52.98177096188778}, {'x': 21.0, 'y': 55.86397854474082}, {'x': 31.0, 'y': 59.832902878704665}, {'x': 39.0, 'y': 60.994526725356494}, {'x': 54.10284436110869, 'y': 68.0}, {'x': 54.046665541361946, 'y': 193.0}, {'x': 52.12380228692915, 'y': 203.0}, {'x': 51.10283931955072, 'y': 233.0}, {'x': 54.01215736711536, 'y': 257.0}, {'x': 54.06960009285053, 'y': 314.0}, {'x': 50.00975058378319, 'y': 322.0}, {'x': 54.019254253358504, 'y': 327.0}, {'x': 54.00828121208217, 'y': 334.0}, {'x': 53.01847237239503, 'y': 345.0}, {'x': 50.0, 'y': 350.0678690503257}]}, {'x': 220.5, 'y': 159.5, 'width': 17.0, 'height': 29.0, 'confidence': 0.016441255807876587, 'class': 'traffic-light', 'points': [{'x': 226.0, 'y': 171.16769038865795}, {'x': 218.0, 'y': 171.12669863641383}, {'x': 211.88245329010874, 'y': 165.0}, {'x': 212.0, 'y': 145.60389001002468}, {'x': 229.3406804695391, 'y': 146.0}, {'x': 229.2320336475416, 'y': 167.0}, {'x': 226.0, 'y': 171.16769038865795}]}]}
Roboflow and Jetson Orin Nano
The combination of the Jetson Orin Nano and Roboflow opens up use cases for computer vision at the edge. Whether you're building a home security system, a wildlife monitoring device, an industrial inspection tool, or a train detection solution, this robust duo has you covered.
Moreover, the small footprint and low power consumption of the Jetson Orin Nano means that you can run computer vision models in almost any environment. Whether you're at the edge or in a constrained setting, deploying a powerful computer vision system is just a few steps away.
Cite this Post
Use the following entry to cite this post in your research:
Paul Guerrie. (Jul 24, 2023). How to Deploy Computer Vision Models to Jetson Orin Nano. Roboflow Blog: https://blog.roboflow.com/how-to-deploy-to-jetson-orin-nano-computer-vision/
Discuss this Post
If you have any questions about this blog post, start a discussion on the Roboflow Forum.