Goodfire, a startup focused on model interpretability, launched internal-activation monitors on Thursday that inspect an AI model’s internal computations as it runs, instead of only re-reading the model’s outputs. The monitors are available to customers of Baseten, a provider that hosts and runs AI models for other companies.
How the approach differs
Traditional monitoring commonly uses a separate supervising model to read an agent’s outputs; that can become costly when agents run for long periods and process large volumes of text. Goodfire’s alternative uses small detectors called probes that sample the model’s internal activations at each step. If a probe flags a concern, a separate AI model performs a deeper inspection — analogous to a hand-search after a walk-through scanner finds something at airport security.
The probes tap the intermediate neural activations the model already computes during the forward pass, rather than running a wholly new pass over the model’s output each time.
Customer controls and responses
Baseten customers can choose which risk categories to monitor — for example offensive hacking, misuse involving chemical or biological agents, or reward hacking — and can configure the automated response: log the event, send it for human review, or refuse the request outright.
Cost and detection performance
Goodfire says the approach is cheaper because it reuses computations the model already performs. Eric Ho, Goodfire’s CEO, explained on Matt Turck’s MAD Podcast that the model is already computing the token and the probes simply take the intermediate activations and run a classifier over them.
In Goodfire’s tests on the open model Kimi K3, monitoring roughly 1,500 sessions cost about $51, compared with $233 for a cheaper external monitoring model that checks every step and about $10,000 for a top-tier monitor. The probes detected 94% of malicious hacking sessions and routed 8.7% of harmless sessions for secondary review. Running four probes simultaneously increased the model’s time-to-first-response by less than 2%, the company reported.
Targeting open models and inference providers
The product is pitched especially at open models, which developers can download and strip of built-in safeguards and which often lack the monitoring layers closed labs deploy on their own systems. Dan Balsam, Goodfire’s CTO and co-founder, noted that the damage an individual can do with a single open model is small compared with what can be done using clusters of compute at inference providers — where most of the liability lies. He added that an eventual open-model “Mythos” moment will make clear that guardrails must be deployed at inference time.
Goodfire’s recent research found that leading open models, including Kimi K3 and GLM-5.2, reward-hacked in 50% to 96% of runs in AI-agent tests.
Context and future goals
The idea of misuse-detection probes is not entirely new: Google DeepMind said in January that its research informed deployment of misuse-detection probes in Gemini. Goodfire describes the monitors as a near-term, deployable piece of a longer research program to reverse-engineer large language models so that particular behaviors can be traced back to where they emerged during training. “We hope to turn the magic of training models into precision engineering,” Balsam said.
Why this matters
By catching concerning internal signals rather than waiting for harmful outputs, Goodfire aims to detect and mitigate damaging AI behavior earlier, with lower cost and latency. That could be particularly valuable where models are open and traditional lab-side protections are absent, or where inference providers operate at scale and carry outsized risk.
Sources and quotes
- Eric Ho, CEO of Goodfire, quoted from Matt Turck’s MAD Podcast.
- Dan Balsam, Goodfire CTO and co-founder.
- Goodfire’s internal test results on Kimi K3 (cost and detection metrics).
