Heather Ceylan, Chief Information Security Officer at Box, warns that identity and permissions remain a necessary foundation but are no longer sufficient to secure enterprise environments against autonomous AI agents. Traditional access controls determine what resources an actor can reach, but they do not govern how an agent behaves once it begins operating on its own. An autonomous agent can turn legitimate access into unintended actions in seconds.
Limits of traditional access controls
Access controls were designed for humans and human workflows: a person with lingering permissions to an old folder will rarely comb through its contents. An autonomous agent, however, will explore every permission it has, surfacing forgotten misconfigurations and stale permissions at machine speed. That dramatically increases the chance of discovering paths to sensitive data that a human would not find.
Recent incidents show this risk in practice: models have escaped intended sandboxes, reached systems they were not scoped to touch, or read content they weren’t supposed to access. In each case, an agent found a reachable route to data and exploited it.
Task‑scoped permissions and governing execution
Problems multiply when agents are granted broad, standing permissions to support wide workflows. An agent might legitimately need to call dozens of tools and touch files across departments over the life of a task. Granting all of that access at once expands the blast radius if any single step goes wrong. Ceylan argues for permissions that change based on what the agent has been asked to do and when it needs to act: if a particular step requires only two tools, scope the permissions to just those two.
Security must shift from simply checking access to deciding whether an agent should perform a specific action at a specific step. An agent may legitimately have rights to a finance folder, yet it should not be allowed to write thousands of files to a public location even if prompted to do so. Prompts alone cannot reliably constrain behavior because they can be altered, injected with instructions, or influenced by read files. Durable controls need to sit at the level of tool calls and the content those calls act upon, defining in advance what an agent may execute regardless of how prompts are manipulated.
Legacy content platforms were not built for agents
Most corporate content that agents touch is unstructured — contracts, policies, customer records — and much of it resides in legacy systems such as network drives, aging ECM platforms, and various SaaS tools designed for human filing habits. These systems typically answer the question “does this person have permission?” and return folder‑level access that may not have been audited for years.
Legacy platforms often lack the metadata or classification required for enforcement, and their logs aren’t granular enough to show exactly what an agent read. Adding an AI connector to that stack doesn’t fix the underlying gaps; it only hands agents the same blind spots at machine speed.
"Every agent action eventually resolves to content. If the content layer can't tell you what it's holding, who it belongs to, and what should never leave it, there's nothing underneath your controls," Ceylan says.
Which actions should require human approval?
Two years ago the prevailing assumption was that security would always mean humans in the loop. As organisations build and operate agents, that assumption has evolved. Box now sorts actions into three tiers: fully autonomous, monitored, and high‑risk actions that require human approval.
- Fully autonomous: reserved for actions that are reversible, bounded, logged, free of untrusted input, and where a mistake isn’t too costly.
- Monitored: actions a team trusts enough to pair with alerting and rollback mechanisms that can detect and undo a problem in flight.
- Irreversible, high‑risk: always routed through a person, for example when an agent requests deletion of a large number of files or wiping a primary folder.
Each team must calibrate the boundaries between these tiers to match its risk tolerance. Box’s approach is to enforce controls in the platform rather than the workflow, applying protections such as data classification, labeling, and expiration up front rather than blocking actions at the end.
"The right configuration should be enforced at the outset, instead of blocking an action at the end," Ceylan says.
Building trust through behavior over time
An access decision can be made once, but trust in an agent’s behavior must be earned over time by observing how it runs, collaborates, and builds on others’ outputs. Published frameworks often lag the technology or are too abstract; Box therefore relies on a smaller set of practical principles: tightly scoped agent identities and actions, clear expectations about rollbackability, three approval tiers, and fast iteration for teams to test and improve.
Visibility is critical, but traditional monitoring tools were built around human baselines and may not detect suspicious agent behavior. Many agents begin as experiments and are not initially included in logging infrastructure. Important signals often appear not as single flagged actions but as chains of activity across systems — when one agent’s output becomes another’s input — and detection for those patterns is still maturing.
Conclusion: visibility and execution controls at the content layer
Governing autonomous agents requires visibility not just into what they can access but into what they actually do, and that visibility must live where the content does. Identity and permissions remain the first layer of defense, but organisations must also scope permissions to tasks, govern execution at the tool and content level, and monitor behavior over time to limit unintended consequences.
This article is sponsored content.
