Research

AI-generated text

NVIDIA reports provisional inference efficiency gains for Vera Rubin

Provisional MLPerf Endpoints v1.0 results show NVIDIA Vera Rubin NVL72 delivering higher inference throughput per kilowatt than NVIDIA GB300 NVL72 across three AI models.

NVIDIA reports provisional inference efficiency gains for Vera Rubin

Provisional MLPerf Endpoints v1.0 results reported by NVIDIA show that the Vera Rubin NVL72 system achieved higher inference throughput per kilowatt than NVIDIA GB300 NVL72 across three AI models. For Kimi K3, the difference was more than tenfold at measurement points that both met a minimum end-to-end average interactivity threshold of 50 tokens per second per user.

The GB300 results have been verified by MLCommons Association, but the Vera Rubin figures are provisional: they await verification and peer review and may change. The GB300 measurements were published on November 4, 2026; the Vera Rubin results for Kimi K3 and Qwen were published on November 9, and those for DeepSeek on November 10.

What the test measures

Agentic AI systems work with conversation histories that grow over multiple turns while generating reasoning, tool calls, and responses. Serving systems must therefore handle many users while remaining responsive.

MLPerf Endpoints v1.0 examines coding and enterprise workflows through API endpoints. During performance runs, generated tool calls and workflow intent are checked against reference values. Separate SWE-bench Verified evaluations assess end-to-end task-solving ability. These checks are intended to assess model quality alongside serving speed.

In the test data, coding histories accumulate over many turns, while enterprise workflows frequently reuse instructions, tool definitions, and business rules.

The energy-efficiency results

In the Kimi K3 comparison, the Vera Rubin measurement point used 32 accelerators and processed 41,414.2 tokens per second at 100.89 kilowatts of operating power, with 1,024 concurrent queries. Average interactivity was 50.4 tokens per second per user. The GB300 measurement point, using 48 accelerators, achieved system throughput of 2,210.4 tokens per second at 60 kilowatts and interactivity of 56.2 tokens per second per user, with 48 concurrent queries.

Both measurement points thus met the common interactivity requirement of at least 50 tokens per second, but they ran at different concurrency levels and accelerator counts. The efficiency ratio divides system token throughput by the stated operating power.

The reported, rounded comparisons across the three models were:

  • Kimi K3: tenfold throughput per kilowatt; threefold interactivity at matched query concurrency.
  • DeepSeek-V4.1-Flash: ninefold throughput per kilowatt; fivefold interactivity at matched concurrency.
  • Qwen3.6-35B-A3B: fivefold throughput per kilowatt; threefold interactivity at matched concurrency.

For the efficiency comparisons, the minimum average interactivity was 280 tokens per second per user for DeepSeek and 410 for Qwen. The interactivity ratios were calculated from separate measurement points at 1,024 concurrent queries for Kimi and DeepSeek and 256 for Qwen. The configurations used in each measurement were not necessarily the same size.

Higher throughput per kilowatt can allow more agents to be served simultaneously within the same power budget while meeting a given responsiveness requirement. At low concurrency, faster responses can reduce waiting times for latency-sensitive tasks.

Hardware and software optimizations

According to NVIDIA, Rubin GPUs combine HBM4 memory, NVIDIA Tensor Cores, and sixth-generation NVIDIA NVLink. The memory supports rapid access to model weights and conversation state, Tensor Cores accelerate matrix operations, and NVLink supports communication between GPUs.

The serving software also reuses previously processed context to reduce repeated computation. Alongside prefix caching and state reuse, NVIDIA Dynamo routing distributes requests among workers. Disaggregated serving allows input processing and token generation to be optimized independently.

The different model architectures also received targeted GPU optimizations. Qwen uses fine-grained prefix matching to reuse conversation history, while its multi-token prediction pairs token proposals with Tensor Core verification. Kimi combines disaggregated serving with input-processing changes that reduce GPU synchronization waits. DeepSeek’s MegaMoE kernel uses Rubin GPU locality and matrix-multiplication instructions.

The provisional results indicate that coordination between hardware, system architecture, and serving software can substantially affect agentic inference capacity. A final assessment of the Vera Rubin measurements, however, still requires verification by MLCommons Association.