AI startup Inherent has developed Faraday, a 27-billion-parameter supervisory model post-trained on Qwen-3.6-27B that leverages OpenAI Codex as a tool to guide larger proprietary frontier models in reproducing missing results from published research. To train and evaluate the system they compiled Replica, a dataset of 100 ML and AI-for-science papers (1990–2026) converted into 310 replication tasks.
System design and training approach
Inherent built a supervisory harness and a relatively small language model that sits above larger frontier models and controls them to improve scientific effectiveness. Faraday itself is a 27B model post-trained on Qwen-3.6-27B and uses a coding agent (OpenAI Codex) as an underlying tool.
To create training and evaluation data, the team converted 100 selected papers into 310 tasks by removing individual key results from the papers; each task requires the AI to autonomously run experiments or computations that fill the omitted results. For each task the authors prompt Claude Opus 4.7 with a meta-rubric to generate a task-specific grading rubric. A Codex-based Judge then provides an overall reward and per-turn credit-assignment weights, which are used in a modified GRPO loop to train the Faraday agent.
Replica dataset and evaluation
Replica comprises 100 machine learning and AI-for-science papers published between 1990 and 2026, transformed into 310 replication tasks by knocking out individual results. The evaluation pipeline uses rubrics generated by Claude Opus 4.7 and a Codex-based judge to supply reward signals and per-step credit for training with a modified GRPO algorithm.
Reported results
According to the authors, Faraday using Codex outperforms standard Claude Opus 4.8 and GPT-5.5 on a subset of replication tasks. Measured by their rubric-based judge, Faraday exceeds the competitors on 73% of in-distribution ML tasks and on 60% of held-out AI-for-science tasks. The paper states that Faraday achieves a comprehensive uplift in performance over the base Qwen model on both training and test tasks.
Significance and implications
The Inherent team frames this work as part of broader efforts to evaluate AI creativity and scientific intuition. They argue that the same abilities that let Faraday fill in vaguely specified details could enable it to design and run its own experiments, which has implications for the possibility of AI-driven advances and recursive self-improvement. The authors suggest that a single post-trained outer agent might be able to track and harness improvements in frontier models over time, enhancing autonomous scientific workflows while also raising strategic questions about future capabilities.
Reference
Training AI Scientists to Replicate Research (arXiv).
