OpenAI has partnered with Ironclad to convert real contracting workflows into research tasks aimed at training and evaluating large language model agents for complex, multi‑step professional work. The collaboration focuses on high‑value tasks that are difficult to automate reliably and that reflect actual customer needs.
Why this matters
The project’s goal is to make AI agents capable of using specialized software to execute multi‑step business processes while maintaining and enforcing an organization’s rules — for example, configuring approval flows, maintaining reusable legal clauses, and verifying that final outputs meet the original requirements.
Ironclad’s role and the research tasks
The first partner in this effort is Ironclad, a company that provides contracting software. Ironclad employees and OpenAI users of Ironclad identified 11 research tasks across legal, commercial, and procurement domains. Examples included:
- setting up nondisclosure agreements (NDAs);
- creating procurement approval processes (such as handling finance approvals above a spending threshold);
- updating reusable legal clauses so they reflect the jurisdiction selected by a requester.
OpenAI estimated that an experienced human user would need roughly 30–40 minutes on average to complete one of these tasks.
Training and evaluation methodology
Ironclad provided hosted product environments where models could practice the tasks. OpenAI researchers:
- defined detailed evaluation criteria (each task was measured against 8–50 criteria depending on complexity) to identify which steps a model handled correctly and where it failed;
- created synthetic training tasks based on representative workflows so models could practice in a controlled way;
- used reinforcement learning with practice and feedback to help the models improve.
This approach—choosing tasks with domain experts, using granular success criteria, and giving models a place to practice—was intended to focus model improvement on real‑world problems important to customers.
Results: GPT‑6 Astra versus GPT‑5.6 Sol
GPT‑6 Astra was the first frontier model trained on the Ironclad tasks. On the research evaluation covering the 11 tasks, Astra’s average score was 55.0%, compared with 41.6% for GPT‑5.6 Sol. OpenAI reports that estimated average time per attempt fell from 37.0 minutes for Sol to 19.2 minutes for Astra — roughly a 48% reduction in simulated attempt time. The organization also reports an internal model used during Astra’s development achieved 63.7% on the same tasks.
For the comparison, OpenAI used the model settings that produced the highest scores for each: "Max reasoning" for Astra and "High reasoning" for Sol.
Demonstrated differences
OpenAI provided two short clips illustrating how each model handled the same research task. In those examples, Astra met about 94% of the task criteria in an estimated 20 minutes, while GPT‑5.6 Sol met about 85% in an estimated 32 minutes.
Limitations and safeguards
OpenAI stresses that these results apply to the 11 selected research tasks, not to all Ironclad workflows. The reported times are simulated estimates based on assumed model processing and generation speeds and are not measured customer time savings.
The simulated training tasks were created from contracts publicly available in the U.S. SEC’s EDGAR database after applying filters intended to remove personal information. OpenAI says it did not use OpenAI customer data, OpenAI’s internal contracts, or nonpublic Ironclad customer data or contracts for training or evaluation.
Ironclad’s involvement highlights a practical challenge customers face: contracting processes must accommodate exceptions while preserving business rules. If an AI agent loses track of a rule mid‑workflow, that limits what tasks a vendor can safely delegate to it. Because of that risk, OpenAI underscores that human oversight remains important and that a full contracting platform is still essential.
Invitation to other software partners
OpenAI is inviting a small number of software companies to work directly with its research and engineering teams on professional tasks that current agents cannot yet complete reliably. Prospective partners are asked to provide a concrete example of the task, evidence of current failures, and the criteria they would use to judge success. They should also bring people who deeply know the work, a secure testing environment, and data that can be used safely for research.
Summary
The OpenAI–Ironclad collaboration shows that task‑focused training, detailed success criteria, and practice environments can improve a model’s accuracy and efficiency on complex contracting tasks. The reported gains are limited to the specific research tasks and simulated timing assumptions; human oversight and comprehensive contracting platforms remain necessary to manage risk.
