Faraday Leads GPT-5.5 and Claude Opus 4.8 at Replication

Inherent’s Faraday research agent, built by DeepMind alumni, led OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.8 on paper-replication tasks, according to the technical paper published by Inherent. The result comes from Replica, a benchmark requiring an agent to reconstruct missing scientific results under fixed compute and time limits.

For developers building long-running research or coding agents, the important detail is the system design. Faraday is a 27-billion-parameter model post-trained from Qwen-3.6-27B, with a supervisory role that separates research planning from implementation.

Replica Benchmark Results

Replica covers 100 machine learning and AI-for-science papers published between 1990 and 2026. Those papers are converted into 310 figure-replication tasks by removing selected experimental results or plots. The agent must infer what to run, write and execute the necessary code, and reproduce the missing data without seeing the original figure.

Evaluation split	Faraday result
In-distribution machine learning tasks	73% task-level dominance
Held-out AI-for-science domains	60% task-level dominance

The held-out tasks span materials science, structural biology, weather forecasting, and meta-learning. Faraday also produced substantial gains over its Qwen-3.6-27B base model and surpassed the general-purpose frontier systems evaluated in the study, GPT-5.5 and Claude Opus 4.8.

The benchmark measures more than code generation. A successful run requires choosing useful experiments, managing a limited budget, diagnosing failed attempts, and deciding when the evidence is sufficient. This makes Replica closer to an agent evaluation than a conventional question-answering test. Teams developing similar systems can use the principles behind evaluating AI agents to separate planning quality from raw model capability.

Supervisor and Worker Separation

Faraday uses an orchestrator-worker architecture. The supervisory model acts as the research lead, forming hypotheses, creating an experimental plan, evaluating intermediate results, and delegating implementation to coding agents. OpenAI Codex performs the underlying coding work.

This division gives the system two distinct optimization targets. Faraday must decide what experiment deserves compute, while Codex turns that decision into executable code. The design resembles broader multi-agent coordination patterns, but the roles are asymmetric rather than interchangeable.

Long-horizon reinforcement learning uses a modified form of Group Relative Policy Optimization (GRPO). Because a final figure may appear only after many tool calls and failed experiments, Inherent adds turn-level credit assignment. Intermediate decisions receive evaluation, helping training distinguish productive research steps from sequences that reach a result by inefficient trial and error.

Automated Evaluation and Training Signals

Faraday’s training loop relies on two judging components. Claude Opus 4.7, prompted with a meta-rubric, generates task-specific grading rubrics. A Codex-based Judge then scores the resulting work, with the goal of producing lower-noise rewards aligned with human expert assessment.

This setup makes evaluation part of the agent architecture rather than a final reporting step. It also introduces an important engineering dependency: the quality of the reinforcement signal constrains the behaviors the system learns. Developers using model-based grading should treat LLM-as-judge evaluation as a measurement system that requires calibration, task-specific criteria, and checks against human review.

Research Agent Economics

Faraday’s result points toward specialization through post-training and tool orchestration. A 27B supervisory model can compete with larger general-purpose systems on a narrow, demanding workflow when the benchmark rewards planning, iteration, and budget management rather than isolated language performance.

Inherent frames replication as a foundation for autonomous scientific discovery. Reproducing an existing result gives an agent practice in developing research taste, selecting experiments, and scoping work under constraints. Those capabilities are prerequisites for systems intended to propose and test novel hypotheses.

If you are building a research agent, benchmark the full scaffold, not only the underlying model. Measure experiment selection, tool-use efficiency, recovery from failed runs, and final reproducibility separately, then validate automated scores against expert judgments before using them as reinforcement-learning rewards.