Egiziago Cioffi, IT and Enterprise Architect and CEO of SynSphere Italia — a Microsoft partner based in Milan — built an Azure OpenAI email assistant himself. He wrote the indexing job, configured the Azure OpenAI retrieval pipeline, and connected it to SharePoint. His team’s evaluations and unit tests all passed; the assistant auto-resolves roughly 60% of inbound customer email, Cioffi said.
The production failure surfaced when Cioffi ran the same questions with a low-privilege account that had previously been asked by a high-privilege account. The assistant returned SharePoint content the requesting user could not have opened directly in SharePoint. Retrieval logs captured this discrepancy even though the evaluation scores showed the system as successful.
Why the gap existed
Azure AI Search currently ships query-time document-level ACL trimming using Entra-backed tokens (available in preview since May 2025), and SharePoint ACL sync followed in a later preview. The SharePoint ACL preview can ingest site-group metadata via the spg: prefix in the 2026-05-01-preview API, but Microsoft’s documentation indicates reliable enforcement at query time is documented only for Entra-backed principals. The preview REST API and SDKs do not cover every agent deployment path.
Custom RAG (retrieval-augmented generation) pipelines can bypass Azure AI Search entirely and thus index under a broadly privileged service account without a query-time entitlement check unless the developer implements one. Cioffi’s deployment used such a custom pipeline, so the platform’s native trimming layer never ran during retrieval.
How widespread is the problem
Straiker’s red team executed more than 1,700 successful exploit attempts against production agents and published the findings in the inaugural STAR Labs Threat Report in July. Their research reports that 91% of successful attacks on the productivity agents they studied ended in silent data exfiltration without detection; no malware was required and there was typically no lateral network movement. The report does not specify how many of those successes were due specifically to retrieval-time entitlement failures as opposed to prompt injection, tool abuse, or other attack types.
Separately, the U.K. AI Security Institute (UKASI) documented 19 unsanctioned agent actions during a July 25–28 cyber evaluation and published an incident report on August 4, 2026. That exercise ran in a permissive test environment with cyber classifiers disabled and internet access enabled; it demonstrates agents acting outside intended scopes and a failure to contain that deviation before damage — a containment failure that overlaps conceptually with Cioffi’s case because both lack reliable runtime scope checks.
The security perspective on the failure
Adriel Desautels, founder and CEO of Netragard, framed the issue as a collapse of authorization boundaries: if an indexing identity (NHI) has broad read privileges, the indexed data will include high-privilege content. If the app does not enforce identity-aware retrieval, a normal user with lower permissions can query the app and receive restricted data, effectively collapsing authorization to the lowest privilege with search capability.
In practice, agents often run as a single long-lived non-human identity with wide permissions. Evaluations tend to focus on answer correctness, relevance and task completion, and rarely assess prompts, outputs, transcripts, memory or logs where sensitive content can be read or injected. That evaluation blind spot is what allowed Cioffi’s system to pass tests while still leaking restricted content at query time.
The fix Cioffi applied and its trade-offs
Cioffi implemented a query-path filter that enforces entitlement decisions inside the retrieval flow: before the model receives a chunk, the filter checks whether the requesting user’s SharePoint permissions allow access to the document. This is a query-time control rather than an index-time one, so content the user cannot open in SharePoint is excluded from the model’s context window.
The filter narrowed the assistant’s reachable content, but with the filter in place the assistant still auto-resolves roughly 60% of inbound email, according to Cioffi. He did not provide a pre-filter auto-resolution percentage for comparison. The qualitative trade-off is that some source material the assistant previously used to answer questions is now omitted when the requesting user lacks permission — the cost of enforcing the permission boundary.
Whether retrieval-time entitlement filtering is acceptable depends on the sensitivity of indexed content, permission variance across users, and whether the deployment can tolerate unanswered queries when the filter blocks a needed chunk. Cioffi’s case demonstrates the gap exists in custom Azure OpenAI pipelines, that answer-quality evaluations do not catch it, and that a query-path filter closes the gap at a describable trade-off.
Identity governance versus retrieval-time controls
Identity governance platforms manage which service accounts exist, what they can access, and token lifecycle (expiration). Examples of industry moves in identity and security: CrowdStrike announced its $740 million acquisition of SGNL on January 8, 2026 and closed the deal on February 20, 2026; Palo Alto Networks announced its $25 billion acquisition of CyberArk in July 2025 and closed that deal on February 11, 2026. These transactions underscore identity as a strategic security pillar at major vendors.
However, governance of credentials does not replace retrieval-time entitlement checks. All credentials in Cioffi’s chain were legitimate and properly managed; the knowledge base was correctly indexed. A low-privilege user queried the assistant and received answers drawn from the full index. No credential misuse occurred, so nothing flagged the retrieval. Azure AI Search’s native query-time ACL trimming and Cioffi’s query-path filter operate at the same layer — the retrieval permission boundary — and both are needed alongside identity governance to close the overall risk.
A one-question, one-test recommendation
Security teams can and should ask: whose permissions does each AI retrieval system use when it fetches content? If the deployment uses Azure AI Search with the SharePoint indexer and Entra-backed principals, verify that query-time ACL trimming is enabled and that the user population does not rely on SharePoint site groups. If a custom retrieval pipeline is used, the entitlement check may be missing entirely.
Start with a practical test: use two accounts (one high-privilege and one low-privilege), ask the same question with both, and compare the assistant’s outputs to what the low-privilege account can access directly in the underlying system. If the assistant returns more than the account’s direct access allows, the retrieval permission boundary is not enforced at query time. The test requires two accounts and roughly thirty minutes and produces an answer an evaluation score cannot replicate.
Conclusion
Cioffi built his agent on a custom Azure OpenAI pipeline that bypassed Azure AI Search’s native ACL trimming. His team’s standard evaluations measured answer quality and passed, but they did not check whose permissions the assistant used at retrieval time. A simple two-account comparison performed before deployment can reveal whether a system enforces the retrieval-time entitlement boundary.
