A workshop attended by more than 50 academic and industry contributors at the Google Contextual Agent Privacy and Security (CAPS) Workshop in late 2025 produced the report "Open and Emergent Problems in Agentic Privacy and Security: A Contextual Angle." The report catalogues foundational privacy and security challenges faced by autonomous agents and calls for coordinated defenses at the system, model, user, and ecosystem levels.
Why agents differ from traditional software
The report emphasizes that useful agentic AI often requires access to personal data and the ability to take consequential actions across varied contexts. That combination demands different approaches than those used for deterministic software. The authors identify three critical dimensions that create research and security challenges:
- Unstructured interfaces and input ambiguity: Agents accept instructions in formats such as natural language and images, complicating definitions of "expected behavior" and increasing vulnerability to adversarial manipulations like prompt injection.
- Probabilistic control flows: Generative planning produces probabilistic execution paths that are difficult to cover with traditional testing methods.
- Autonomy and delegation: As agents carry out longer tasks and delegate sub-tasks to other agents, traditional user oversight becomes less effective, increasing the risk of confirmation fatigue.
The contextual lens — Contextual Integrity
The report argues that trustworthy agents must reason about social norms and the appropriateness of actions for the specific context in which they operate. It draws on the theory of Contextual Integrity (CI), which defines privacy as "appropriate information flow" governed by social norms rather than solely secrecy or control.
A context-specific informational norm in CI is defined by three elements:
- actors (who sends and who receives information, and about whom),
- types of information (e.g., medical or financial records),
- transmission principles (rules governing flow, such as confidentiality or reciprocity).
The report extends CI from information sharing to contextual security — judging whether an agent's action is appropriate. Anchoring privacy and security in CI enables systems to evaluate whether a proposed action is socially and contextually acceptable before execution.
Operationalizing Contextual Integrity
According to the authors, modern LLMs create an opportunity to produce machine-readable, genuinely context-dependent policies. Historically, a semantic gap existed between high-level contextual norms and low-level system permissions. For example, "protect my data while organizing my travel for a conference" requires specific rules about what user information can be shared for flight bookings, visa applications, and organizer communications.
Manual permissions and expert-written policies do not scale to a future where agents autonomously perform multiple complex, long-running tasks. The report argues that as language models learn CI, they can help bridge that gap. The authors propose a contextual policy engine integrated into a supervisory layer that monitors and enforces appropriateness. This engine would include a dynamic policy generation loop capable of producing real-time, tailored policies for user requests and open-ended contexts, including new tools discovered at runtime. Such a system could assess whether a requested data flow is appropriate before any information leaves the user's workspace.
A multi-layered approach to agentic privacy and security
Combining contextual policy engines with advances at the model and user levels yields a multi-layered strategy. The report outlines innovation opportunities across the stack:
- System-level sandboxing: Execution environments can impose additional guardrails to limit the impact of agent or model failures, enable detection and response, dynamically limit agentic capabilities, establish agent identity, and authorize or revoke data access based on context changes.
- Model-level reasoning: Models should be able to disambiguate under-specified user prompts and reason about appropriateness under changing contextual norms, for example when deciding whether to share user data.
- User-centric controls: Because the traditional "Notice and Choice" framework relies on foreseeability, the report explores shifting interfaces toward dynamic, contextual, and personalized control mechanisms that match user mental models.
- Multi-agent interactions: Effective guardrails are needed to prevent agents from colluding and violating contextual norms when collaborating on complex tasks.
- Ecosystem governance: Governance mechanisms must support collecting and sharing contextual norms, resolving conflicts between norms, negotiating changes, and verifying compliance across domains and entities.
Measuring privacy and security dynamically
The report calls for new safety evaluation approaches suited to highly autonomous, multi-agent systems. It highlights the need for standardized, multi-agent benchmarks — dynamic "Agent Gym" environments where researchers can safely simulate complex, cascading interactions over extended periods. Open-source sandboxes would help establish a shared baseline for privacy, security, and safety across academia and industry.
A call to action
The authors stress that building a safe, secure, and trustworthy agentic ecosystem is too large a task for a single discipline or organization. They invite the broader research community across academia, government, civil society, and industry to contribute ideas and collaborate on the contextual foundations required for scalable, real-time mechanisms that judge the appropriateness of agent actions.
Acknowledgements
Lillian Tsai served as co-primary author and Sarah de Haas coordinated the workshop, report writing, and blog post. Kassem Fawaz, Stefan Mellem, Helen Nissenbaum, and Nina Taft contributed to conceptualization and multi-section writing. Section leads included Sebastian Benthall, Madiha Zahrah Choksi, Seliem El-Sayed, Ferdinando Fioretto, Adrià Gascón, Ashish Hooda, Ivan Petrov, Francesco Pinto, Octavian Suciu, Trishita Tiwari, Harold Triedman, Ren Yi, and Wen Zhang. The report acknowledges contributions from over 50 researchers, academic partners, and industry leaders, and notes support from Daniel Ramage, Corinna Cortes, Yossi Matias, Pushmeet Kohli, Hank Levy, Raluca Ada Popa, Amanda Walker, Brendan McMahan, and James Manyika.
The full technical report is available from the authors for readers seeking more detail.
