Nvidia researchers published new findings on Friday arguing that the harness — the software and runtime scaffolding surrounding a model — can be far more important than the underlying language model when asking an AI to carry out long‑horizon tasks.
Key facts and results
- Using a custom, memory‑aware harness that includes a supervising, boss‑like component, Nvidia researchers report that Claude Opus 5 achieved a 100% score on the interactive reasoning benchmark ARC‑AGI‑3. ARC‑AGI‑3 consists of a set of 2D games with no instructions; the model must discover how to play and win.
- With the standard harness (i.e., without Nvidia’s customizations), Opus 5 scored 30% — the best result among all models tested in that baseline condition.
- OpenAI initially scored poorly on ARC‑AGI‑3 (under 10%) and then found that tweaking two harness settings tripled its models’ scores. However, none of OpenAI’s models reached the 100% result reported by Nvidia’s team.
What the harness does and why it matters
Adel El Hallack, vice president of product in Nvidia’s AI unit, told TechCrunch that people often treat an agent like the model’s API, but an agent actually includes the model plus the scaffolding around it — the harness — together with the runtime, tools, skills and libraries the model can access.
Long‑horizon tasks require chaining many decisions over time, sometimes across days, to produce finished work. The core difficulty is keeping the agent focused, preserving memory and context, and avoiding irrelevant or destructive exploration. Nvidia’s research highlights that a harness can provide those capabilities.
The supervising component acts like a CEO: it nudges the working agent back on track if it drifts toward dead ends or repeats unproductive paths.
Tools, openness and ecosystem implications
Nvidia’s enhanced harness in the study is called Agentic Variation Operators (AVO). It is a research harness rather than a commercial product. Nvidia also offers many building blocks for harnesses under the NeMo brand; some of that technology is commercial, and much is openly available.
The findings align with other recent work showing the harness can shape both performance and cost. In July, Databricks published research demonstrating that harness choice can dramatically influence AI costs; Databricks CEO Ali Ghodsi told TechCrunch that using different harnesses with the same model can double costs.
Nvidia’s broader point is that open, configurable harnesses — like open models — give users more control. "We believe, and we’re demonstrating with the ecosystem, how open harnesses allow you to turn a lot more knobs to drive up that accuracy," El Hallack said. He added that an open agent stack with control across harness, infrastructure and runtime is necessary to advance the ecosystem securely.
Practical significance
The study reinforces that while model selection matters, the surrounding system — memory handling, feedback loops and supervisory layers — is a major determinant of success for multi‑step, long‑horizon problems. For applications that require reliable, multi‑stage workflows (for example document editing automation or interactive decision tasks), designing the right harness may be as important as, or more important than, switching base models.
Further developments in harness design, and the tradeoffs between open and closed solutions for performance, cost and security, will be important to follow.
