NVIDIA released Dynamo 1.0 at GTC 2026 as free, open source software designed to orchestrate inference across GPU clusters. The company calls it the "operating system for AI factories." AWS, Microsoft Azure, Google Cloud, and Oracle Cloud Infrastructure are all adopting it. So are CoreWeave, Together AI, Nebius, Alibaba Cloud, and DigitalOcean. Cursor and Perplexity are already running it in production. ByteDance, Meituan, PayPal, and Pinterest are on the enterprise adoption list.
That adoption spread is the story. A single inference runtime touching all four hyperscalers, the major inference-native providers, and a cross-section of consumer-facing AI companies does not happen because the software is interesting. It happens because the problem it solves — scheduling tokens across distributed GPUs without wasting memory or compute — is now expensive enough that everyone needs the same answer.
The scheduler layer nobody had
Until now, inference orchestration has been a patchwork. Teams stitched together vLLM or TensorRT-LLM for the execution engine, custom scripts for scaling, and ad hoc memory management for the KV cache that balloons when models handle long-context or multi-turn agent conversations. Dynamo packages that stack into three modules:
KVBM handles memory management for the key-value cache. Instead of pinning cache to the GPU that generated it, KVBM can offload to lower-cost storage when context is not actively needed and pull it back when a follow-up request arrives. For agentic workloads — where a model might handle a multi-step task over minutes, not milliseconds — this is the difference between holding expensive GPU memory hostage and freeing it for other work.
NVIDIA NIXL moves data between GPUs. In a disaggregated setup where prefill and decode happen on different hardware, the bottleneck is often the transfer, not the compute. NIXL is NVIDIA's purpose-built transport for that hop.
NVIDIA Grove handles scaling. The pitch is simpler cluster management: adding or removing GPUs from an inference pool without manual rebalancing.
Why NVIDIA is giving this away
Jensen Huang's framing was direct: "Inference is the engine of intelligence, powering every query, every agent and every application. With NVIDIA Dynamo, we've created the first-ever 'operating system' for AI factories."
The business logic is straightforward. NVIDIA sells GPUs. The harder it is to run inference efficiently, the more GPUs you need — but past a certain pain threshold, customers start looking at alternatives: custom ASICs, AMD, or just running fewer queries. An open source inference OS that makes Blackwell GPUs dramatically more efficient (NVIDIA claims up to 7x) keeps the hardware moat intact by making the software layer a shared commodity that happens to run best on NVIDIA silicon.
This is the CUDA playbook applied to inference. Own the layer that developers build on, make it free, and collect on the hardware.
Agentic routing is the design bet worth watching
The most forward-looking piece of Dynamo is its traffic routing for agentic AI. Traditional inference treats each request as independent: a prompt arrives, gets processed, and leaves. Agentic workloads break that model. An agent might issue a chain of five tool calls, each one building on the last, with shared context that needs to persist across the sequence.
Dynamo's router can direct follow-up requests to GPUs that already have the relevant KV cache loaded. That avoids recomputing context from scratch on every hop — a cost that scales linearly with conversation length and can dominate total inference time for long-running agents.
This is where the integration list matters. Dynamo works with LangChain, SGLang, vLLM, LMCache, and llm-d. If your agent framework talks to any of those backends, Dynamo sits underneath and handles the routing. NVIDIA is also contributing TensorRT-LLM CUDA kernels to the FlashInfer project, which means the performance optimizations flow into the wider open source inference ecosystem, not just NVIDIA's proprietary stack.
Who is actually running it
The production deployments are more telling than the partnership announcements:
- Cursor and Perplexity are AI-native companies where inference cost is a direct line item on the P&L. If Dynamo saves them meaningful GPU spend, it validates the performance claims in a way that benchmark slides cannot.
- Baseten, Deep Infra, and Fireworks are inference-as-a-service providers. Their entire margin depends on squeezing more tokens per GPU-second. Adoption here means the economics actually work.
- ByteDance, PayPal, Pinterest, and Instacart are large-scale consumer platforms where inference volume is enormous and cost sensitivity is real.
The missing names are also informative. No mention of Anthropic, OpenAI, or xAI — companies that have built proprietary inference stacks and are unlikely to cede that layer to NVIDIA.
The cost line moves before the capability line
The 7x performance improvement on Blackwell is a headline number, and like most headline numbers it comes with asterisks — specific workloads, specific configurations, specific batch sizes. The more durable impact is what happens to the baseline cost of running inference at scale.
If Dynamo becomes the default scheduler for multi-GPU inference clusters — and the adoption list suggests it is heading that direction — then the cost floor for serving a token drops for everyone running NVIDIA hardware. That changes the math for which applications are economically viable, which agent architectures are practical, and how aggressively companies can offer AI features without burning cash.
It also means NVIDIA is embedding itself one layer deeper into the inference stack. Today they sell the chips. Now they are defining how work gets scheduled across those chips. The next question is whether Dynamo's design assumptions — disaggregated prefill and decode, KV cache offloading, context-aware routing — become the default architecture for inference, or whether alternative approaches emerge from the companies that chose not to adopt it.
The layer that locks in the hardware
NVIDIA framed Dynamo as an open source gift. The economics say otherwise. Every cluster running Dynamo is a cluster optimized for NVIDIA GPUs, trained on NVIDIA's scheduling assumptions, and integrated with NVIDIA's data movement primitives. The software is free. The lock-in is architectural.
That is not a criticism — it is the same strategy that made CUDA dominant, and it works because the software genuinely solves a real problem. The question for the rest of the industry is whether a shared inference OS controlled by the GPU vendor is a net positive (lower costs, faster iteration, common tooling) or a dependency that gets more expensive to exit over time. For now, the adoption numbers suggest most companies have decided the benefits outweigh the risk.