NVIDIA Releases DynoSim Simulation Framework, Shifting AI Serving Stack Optimization from Hardware Trial-and-Error to Simulation-First

DynoSim is a 'digital twin' of the NVIDIA Dynamo serving stack, built on a discrete-event simulation (DES) architecture. Its core is a composable design where components like workload replay, single-engine simulation (with scheduler behaviors for backends like vLLM/SGLang), Router, Planner, and KVBM run as parallel actors on a unified virtual timeline.

The framework leverages AI Configurator (AIC) for hardware-informed forward-pass timing and combines scheduler simulation to capture system effects like queuing and batching on metrics like TTFT under high concurrency. It is extremely fast, capable of replaying tens of thousands of requests at ~1500x real-time on a MacBook Air.

In practice, DynoSim enables systematic search over deployment knobs (e.g., TP shape, worker count, routing policy) and serves as a scoring function for 'autoresearch'-style algorithmic optimization of core components like Router cost functions, Planner heuristics, and cache policies. The blog uses Planner autoscaling as an example, demonstrating how simulation rapidly evaluates the impact of scaling intervals and cold-start times on cost and SLA.