Up to 30x More Work Per Watt: NVIDIA Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents

NVIDIA Vera Rubin NVL72 delivers up to 30x more agentic work per megawatt than GB300 NVL72 in SemiAnalysis AgentX measurements.

Updated October 28, 2026 at 3:01 AM ET

According to NVIDIA, NVIDIA Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and up to 45x lower cost per million tokens than GB300 NVL72 on the SemiAnalysis AgentX workload. A separate cost comparison from SemiAnalysis puts Vera Rubin NVL72 at about 67x the total throughput per total cost of ownership under its specified comparison.

Those efficiency gains matter for companies deploying agents for coding, customer service, research and other production tasks. Agent sessions consume much more capacity than a simple chat request, so the hardware and serving software behind each agent affect operating cost, available capacity and the pace at which a business can expand.

What does the new efficiency data mean for AI agents?

OpenRouter data cited in the source article shows that agentic AI workloads consume about 15x more tokens than a simple chat request. The increase comes from the number of steps involved in completing a task and the context carried between those steps.

An agent researching a company for an investment decision, for example, may query financial databases, search news and filings, invoke a sub-agent for peer comparisons and model valuations, and then synthesize the results into a recommendation. Each stage adds tokens that become inputs to the next stage.

Chat and document summarization workloads generally use sequences of 1K to 8K tokens. Agentic sessions can reach hundreds of thousands of input tokens, and request lengths vary considerably. Infrastructure must therefore handle growing context, repeated tool calls and the spawning of sub-agents while controlling the amount of power and computing capacity required for each unit of work.

How much more efficient is Vera Rubin NVL72?

The SemiAnalysis AgentX workload uses recorded real-world agentic coding sessions with context growth, tool calls and sub-agent spawning preserved. It measures the full workflow rather than a single inference request.

On that workload, the NVIDIA Blackwell platform leads across several agentic models, including Kimi K3, MiniMax M3, GLM5.3, Qwen3.5 and DeepSeek V4 Pro. With DeepSeek V4 Pro, GB300 NVL72 delivers up to 15x better throughput per megawatt than the NVIDIA Hopper architecture.

Vera Rubin extends the Blackwell advantage across the full performance-versus-power curve. On DeepSeek V4 Pro, it reaches as much as 30x higher throughput per megawatt than GB300 NVL72. The same measurement shows up to 45x lower cost per million tokens.

The latest SemiAnalysis write-up adds a total-cost-of-ownership comparison: “At 170 TPS, On Apples to Apples TRTLLM NVFP4 Dense, Vera Rubin NVL72 delivers ~67x the total throughput per TCO of GB300 Dynamo TRTLLM under the owning cost assumptions.” This figure depends on that comparison, its throughput rate and SemiAnalysis’s owning-cost assumptions.

At the service rates where the write-up says many providers would run the model, between 60 and 100 tokens per second, SemiAnalysis reports a narrower advantage of 1.4x to 3x throughput per total cost of ownership against the latest GB300 TRTLLM configuration. The calculations cover factors such as equipment life, colocation, power, cost of capital, hyperscaler purchasing terms and server-build costs. Users of SemiAnalysis’s model can enter their own compute costs.

NVIDIA DSX MaxLPS power-management technology manages power across the GPU, rack and workload levels. It can provision up to 40% more GPUs within the same megawatt budget, adding further throughput capacity where the power supply is constrained.

How does the platform reduce the cost of agentic work?

How are inference tasks separated and balanced?

Disaggregated serving separates context processing, known as prefill, from response generation, known as decode. Each stage can then scale independently. Rate matching synchronizes the token-production speeds of prefill and decode GPUs so that neither side holds the other idle.

How is model memory and context handled?

Large-scale expert parallelism distributes mixture-of-experts sub-networks across the scale-up GPU domain. Distributed KV-caching extends memory across the same domain, while KV-cache offloading moves less-active context to host and storage so it remains available without being processed again.

KV-aware routing sends requests to GPUs that already hold the relevant cached context. This reduces repeated computation during long agent sessions.

Which hardware and software features support the gains?

Fused CUDA kernels such as MegaMoE combine computation and inter-GPU communication operations in one execution pass, reducing GPU idle time while waiting for data. Fifth-generation Tensor Cores and the third-generation Transformer Engine accelerate prefill and decode. NVFP4 quantization compresses model weights to 4-bit precision, which may reduce memory requirements and increase throughput without sacrificing output quality.

The NVL72 scale-up domain provides the high-bandwidth, low-latency communication needed for large-scale expert parallelism and distributed KV-caching. Sixth-generation NVIDIA NVLink technology and NVLink Switches deliver 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet alternatives.

NVIDIA TensorRT LLM and Dynamo sit alongside optimized CUDA kernels in a software stack designed together with the hardware. These optimizations also apply to GB300 NVL72, so its measured performance can improve alongside Vera Rubin NVL72.

What is included in the wider Vera Rubin platform?

The full Vera Rubin platform uses a seven-chip architecture. Alongside the Vera Rubin NVL72 system, it includes the NVIDIA Vera CPU, Groq 3 LPU, NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX and ConnectX-9 SuperNIC.

The AgentX measurements remain early results and do not yet include Vera CPU performance for tool calling. NVIDIA says Vera Rubin is in full production and scaling across the ecosystem.

What to do now

  1. Measure the complete agent workflow. Use traffic that preserves context growth, tool calls and sub-agent spawning when comparing infrastructure. A single-request benchmark does not represent the token demand of a production agent.
  2. Track throughput per megawatt. If power is constrained, compare the amount of completed agent work supported by the available energy budget rather than processor performance alone.
  3. Calculate cost per million tokens. Include the token pattern generated by the agent, since research, software development, customer service and deep research can produce very different workloads.
  4. Review total-cost-of-ownership assumptions. Compare ownership, colocation, power, financing and equipment-life costs using figures that match the business’s actual deployment model.
  5. Include software optimization in the evaluation. Test the complete serving stack because distributed caching, routing, rate matching and inference runtimes affect the results delivered by the hardware.

FAQ

What does the SemiAnalysis AgentX benchmark measure?

SemiAnalysis AgentX measures performance on recorded real-world agentic coding sessions, preserving actual context growth, tool calls and sub-agent spawning. It captures the full agent workflow rather than a single inference request.

How much more efficient is NVIDIA Vera Rubin NVL72 than GB300 NVL72 on agentic workloads?

On the SemiAnalysis AgentX workload using the DeepSeek V4 Pro model, Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and up to 45x lower cost per million tokens than GB300 NVL72.

Why are agentic AI workloads harder on infrastructure than chat?

Long-context handling is central to agentic AI performance.

Related coverage

This article reports on publicly available information from the sources listed below. We have no reason to believe it is inaccurate, but we have not independently verified every claim, and details may change as the story develops.

Sources


See our editorial disclaimer for how our articles are produced.

🤖
Is your business visible to AI assistants?

Run a free scan to see your AI Visibility Score, SEO rating, and local citation accuracy.

Check Your Score →