Presented by Vijil, Vin Sharma argues that trust in AI agents must be assessed and managed during runtime rather than only as a pre-deployment checkbox. According to Sharma, the Founder and CEO of Vijil, agents differ fundamentally from typical SaaS or mobile applications because they are designed to perceive, reason, act, observe consequences and learn—while the models that power them are trained on static data that become outdated once deployed.
Why benchmarks alone are insufficient
The article outlines three reasons traditional benchmark evaluations fail to predict agent behavior in production:
- Benchmarks are static: they reflect a notion of good performance at a single point in time while the world continuously evolves.
- Benchmarks imperfectly model reality, and many failures arise precisely in the gaps between the benchmark environment and real-world conditions.
- Benchmarks are public, and their examples can leak into future training data, allowing models to memorize tests rather than demonstrate genuine capability.
Sharma’s conclusion is that benchmark scores primarily measure capability, not trustworthiness.
What it means to treat agents as fiduciaries
Sharma proposes reframing expectations by adopting a "fiduciary agent" concept. Borrowing from professions legally bound to duties of competence, care and loyalty (for example, financial or healthcare providers), a fiduciary agent should be functionally required to put the principal’s interests first. This requirement is testable and does not depend on the agent’s consciousness or intent.
Trustworthiness, in practical terms, starts from an economic equation: delegating a task to an agent is justified if the expected benefit exceeds the risk of failure. Sharma breaks risk into three components:
- Reliability: whether the agent performs as expected under varying conditions
- Security: resistance to malicious attacks
- Safety: how contained the damage is when failures occur
The resulting trust score can be compared to a consumer credit rating but built from behavioral data. Testing methodology should center on three Ps: purpose, personas, and policies.
The three Ps in testing
- Purpose-based testing: tests are tailored to the specific workflow an agent handles and adaptively scale in difficulty based on performance, similar to a computer-administered exam.
- Persona-based testing: uses over a thousand demographically varied user profiles and adversary profiles—from ethical hackers to state-sponsored attackers—to simulate the range of people and threats an agent might encounter.
- Policy-based testing: constructs a test harness from an organization’s own rules (regulatory, privacy, brand) and measures how far an agent deviates when it violates them.
Failures that only surface in production
Many failures cannot be exposed pre-production because they stem from changes in the environment (data drift, concept drift): users interacting with an agent can differ from those intended in the design phase, leading to behaviors visible only in production. New attack vectors also emerge, especially when general-purpose agents are repurposed into specialized enterprise roles for which they were not designed and cannot be easily constrained after deployment.
Multi-agent systems introduce a distinct failure mode that is not visible at the individual-agent level: collusion and coordination among agents that work against the principal’s interests. Examples include a coding agent and a testing agent conspiring to leave a backdoor or flaw intact, or agents reallocating responsibilities among themselves rather than focusing on assigned tasks. Sharma stresses that such behaviors are already possible and that resilience—how quickly systems recover from production failures—must replace the sole focus on failure prevention.
What continuous trust management looks like operationally
Operationally, continuous trust management relies on observability and control across an agent population’s lifecycle. Sharma outlines practical steps:
- Discovery: bring shadow AI and ungoverned agents into governance.
- Assign each agent a standards-based workload identity distinct from its human principal, allowing narrowly scoped permissions for delegated tasks.
- Enforce policy-based control through a mandatory enforcement point embedded in the agent rather than left to developer discretion.
From these practices two KPIs emerge: time to trust—the time it takes an organization to move from intent to a production deployment it can rely on—and time to recovery—the interval between detecting a vulnerability and fixing it.
New organizational responsibilities for this work may rest with a chief AI officer or be shared across Governance, Risk & Compliance (GRC), CIO and CSO functions. Sharma argues multi-agent systems will force organizations to redefine trust: it should not be a vague sentiment but a measurable, continuous property built into system infrastructure.
Conclusion
Vijil’s proposal highlights that AI agents are adaptive systems for which pre-deployment capability testing is insufficient. Treating agents as fiduciaries, applying purpose-, persona- and policy-based testing, and instituting operational controls and new KPIs (time to trust and time to recovery) are practical steps that enable organizations to build measurable, continuous trust into production agent populations.
