Researchers at the University of Tübingen, the Max Planck Institute and collaborating institutions have demonstrated a technique that recovers hidden "reasoning traces" produced by state-of-the-art large language models (LLMs) when solving problems. The teams examined models provided via API by OpenAI, Anthropic and Google.
What the researchers found and how the technique works
According to the researchers, some providers send encrypted intermediate outputs from the large model to the user's device. The extraction method relies on the existence of a smaller, cheaper sibling model from the same family that shares the decryption key but has weaker alignment and refusal behavior. Feeding the encrypted intermediate output to that less-constrained smaller model can coax out or partially reconstruct the original model's internal reasoning.
The researchers report that the reconstructed traces at times contained sensitive material, including passwords and API keys, demonstrating that private or confidential data can leak via these intermediate representations.
Practical implications
This leakage arises from architectural choices rather than a single coding bug. Providers mitigate costs by offloading encrypted intermediate data to user devices and offering lower-cost sibling models; those same choices open the avenue for extraction. The research teams say providers issued patches last month to reduce the leakage, but some reasoning still appears to spill out. Fully preventing the problem would require redesigning how APIs operate, which implies substantial infrastructure and business-model implications for providers.
Conclusion
The study highlights that a large model's supposedly private internal inferences can be exposed to the extent that a smaller, more talkative sibling exists in the same model family. The findings suggest that closing this class of leaks cannot rely solely on surface-level fixes and may necessitate architectural changes to how models and their intermediate outputs are managed.
