NVIDIA Jetson Advances Localized Deployment of Open-Source AI Models at Edge

NVIDIA Jetson brings open-source generative AI models to edge devices and robots, enabling low-latency, offline autonomous intelligence. Caterpillar's CES demo of Cat AI assistant uses Jetson Thor with Nemotron voice model and local Qwen3 4B (via vLLM) for real-time cockpit voice interaction.

Hardware spans Jetson Orin Nano 8GB to Jetson Thor, supporting frameworks like TRT, Llama.cpp, vLLM. Performance: Mistral 3 on Jetson Thor reaches 52 tokens/sec, Qwen 3.5-35B-A3B at 35 tokens/sec. Isaac GR00T N1.6 enables end-to-end on-board real-time perception and action.

Ecosystem expands from research (e.g., NYU robotics) to developers, with tools like OpenClaw for building 24/7 private AI assistants, ensuring full local data processing for privacy and zero API cost.