This week’s roundup looks at the structures that make AI capabilities practically useful: session‑spanning memories, maps that preserve places, and the harnesses (or orchestration layers) that help models act in the world. The central question is what should remain outside the model, what can be absorbed or reused, and how to extract extra computation or capability from current weights.
When does a harness help, and when is it overhead?
Engineers diverge: some add more infrastructure around agents, while others strip it away. A harness can be a tiny experimental agent implemented in roughly 25 lines of code, or a system that coordinates tools, permissions, memory and workflows that last for days. Harnesses can make agents usable in practice, but they also add complexity, runtime cost, and may preserve workarounds that newer models no longer need. The practical questions are which responsibilities should live outside the core model, which can be eliminated as models improve, and how to measure whether another orchestration layer is genuinely improving performance rather than just adding overhead.
What should an agent remember?
Memory can serve different ambitions: personalization, learning from experience, and sharing knowledge across an organization. Personalization might mean retaining a preferred report format; learning from experience means updating behavior to avoid repeated mistakes; organizational knowledge requires preserving procedures so other teams can inspect and correct them. Each case requires different guarantees about what should persist after a task finishes and how old information should influence new decisions. Crucially, memory should improve future work rather than propagating yesterday’s errors; testing whether memory helps — without carrying forward harmful habits — is essential.
Can we finally understand what happens inside neural networks?
A renewed debate in interpretability asks whether internal patterns are clearer than previously thought or whether our explanations still fall short of predictive tests. Nick Cammarata reexamined early interpretability work and argued for more optimism about finding meaningful patterns in neurons. Leo Gao responded by demanding harder tests: can proposed explanations predict how a neuron will behave under interventions that change the model’s internal state? The discussion covers superposition (neurons encoding multiple features), circuits (coordinated activation patterns), and the difference between an appealing interpretation and an experimentally validated, predictive explanation. The exchanges include examples with cats, cars, geometric motifs, and experiments that alter Claude’s output without changing the question.
How do robots remember things no longer in view?
3D Gaussian Splatting composes images into scenes made of small, overlapping 3D primitives. Representations like these, and systems built on them (for example GSMem), let previously observed spaces be indexed and searched so a robot can return to a location from a different viewpoint and inspect an object again. Turning a geometric reconstruction into useful spatial memory requires more than photorealism: additional models are needed to predict object dynamics and behavior if we want the system to reason about how things will move or interact.
Improving agents without new models
FOD#172 revisits the harness debate with reader reports and recent research. Some setups benefit from more external support; others perform better when simplified. Results depend heavily on the chosen model, the task, and the available information, which makes modularity attractive because it lets designers try incremental changes. The practical recurring question is: what improvements can we make with models we already have, and how do we know a change actually worked?
Looped Transformers: reusing layers for extra computation
Looped Transformers run the same Transformer layers multiple times to refine internal representations. This creates room for additional computation without proportionally increasing parameter counts, but each extra pass still consumes time and compute. Recent literature maps ten lines of work across reasoning, shared memory, image synthesis, audio, and multi‑agent systems. The central design questions are what should change between passes, what can be reused, and when the model should stop looping so that another pass is genuinely beneficial.
Takeaway
Before automatically chasing the next, larger model, there is substantial value in rethinking the surrounding software and representations: which behavior and data should be externalized, how memories should be stored and validated, whether proposed internal explanations hold under intervention, and when reusing weights can buy useful computation. Answers vary by task and model, and the field needs modular experiments and robust metrics to decide when to add complexity and when to simplify.
