At the AI Engineer New York conference this week, a central theme was how to combine agent autonomy with security and reliability. Presentations from frontier AI labs (OpenAI, Anthropic) and financial organizations (Two Sigma, Bloomberg) converged on this problem.
Emphasis: controls and human judgment, not blanket restrictions
Speakers emphasized that safe autonomy depends less on imposing many restrictions and more on where you place controls and human review points. Well-designed product and interface decisions determine when human approval is required.
OpenAI: a "threshold of usefulness" and traceability
Joseph Kim, Head of ChatGPT for Financial Services at OpenAI, delivered a keynote focused on reliability and what he called a "threshold of usefulness" — the point where a model becomes capable and reliable enough to save time for users. Kim argued that continually improving model capabilities will let financial analysts delegate more tasks to agents.
Kim made forward-looking comments for 2027: he predicted the financial industry will rely less on Excel and PowerPoint, and that agents could become capable of "investment idea generation and trading." He also noted that rising model capabilities are improving traceability. "In finance, you need to be able to check where a number came from and understand the context around it," he said, and product details that integrate review into the experience help turn model capability into usable work.
On when a human must be in the loop, Kim framed the decision as a product design question: how the interface changes and what users must still control, review, or approve are matters teams can start designing now.
Two Sigma: composable controls to enable autonomy
John Feminella, SVP Architecture at the hedge fund Two Sigma, described five controls the firm has implemented to keep agents in check without diminishing model capability. Feminella argued that safety and innovation are not opposites and criticized "approval fatigue," where requiring human sign-off on everything slows innovation and can introduce risk if approvers are not fully attentive.
Two Sigma’s approach uses a set of "guardrails and shared capabilities" that are composable — "each one does one job and together they let us run agents with real autonomy at an appropriate level of safety." One specific control is using auto-mode classifiers that handle routine calls and send a person only the most important requests likely to matter. Feminella said classifiers are useful for ambiguous cases, while deterministic rules suit clear-cut situations.
Anthropic: identity, sandboxing and approval layers
Anthropic presented a similar framework focused on building a "trust" layer into financial agent workflows. Last month, Anthropic announced Claude for Financial Advisors, enabling advisors to connect their systems to Claude and a set of workflow skills.
Belinda Neal, Managing Director of Financial Services at Anthropic, explained that when a model crosses a threshold it can trigger waves of changes in how people work. Keon Shahab, who works on applied AI at Anthropic, described three steps to build the trust layer: defining who the agent is acting as (identity and permissions), sandboxing and establishing an approval layer so someone signs off before action, and ensuring the agent continues to function as variables change over time.
Shahab illustrated the boundary between proposing and acting with a trade break investigation agent that would recommend order amendments for human approval rather than making those changes itself. "By building once with a foundational layer that you trust, you can work at speed to deploy many agents at scale," he concluded.
Bloomberg: enforce deterministic controls outside the model
Andrey Rybka, Head of Architecture & Emerging Technologies in the Office of the CTO at Bloomberg, recounted real incidents where agents caused problems and outlined defenses to prevent them. His list included scoped identities and credentials, controls over tool calls, outbound traffic restrictions, audit trails, budgets, and kill switches.
In one example an agent interpreted embedded instructions in a marketplace review as an order to stock a product. Rybka recommended input validation, prompt-injection defenses, provenance tracking, and human approval for high-value actions. "We want deterministic controls outside the model," he said — check every call, log it, and keep a kill switch.
Go fast and stop safely
Even though frontier labs like OpenAI and Anthropic are building trust frameworks for financial agents, it remains the responsibility of financial firms to wrap controls around their deployments. Two Sigma’s approach — implement controls that enable autonomy rather than block it — captured the conference’s overall message. As Feminella put it: "Why do cars have brakes? It’s not just so that you can stop, it’s so that you can go fast and stop safely."
