A conversation flagged by Anthropic's safety system was escalated to a human reviewer, who judged the exchange to be serious and forwarded the information to law enforcement — resulting in the arrest of a woman in Florida. Authorities have charged her under Florida law with a written threat of violence.
Anthropic uses automated checks that look for keywords and content that may indicate threatening behavior. When the system detects potentially dangerous material, it can trigger human review; depending on the assessed severity, reviewers may pass relevant information to external parties, including the police. Anthropic has acknowledged that in limited emergency circumstances it may share user data if doing so is necessary to prevent death or serious bodily harm.
This incident underscores that a “private chat” within an AI service does not equate to a private, single-user diary: the product operates as an online service with safety protocols that can lead to disclosure of conversations in certain cases. Anthropic says it prefers to err on the side of caution when content could pose a real threat.
The case comes amid broader legal and political pressure on AI companies. The report notes that in the previous month British Columbia sued OpenAI and Sam Altman, alleging the company could have prevented a mass shooting in the province — authorities say the suspect had previously been flagged by OpenAI’s safety team but police were not notified because content did not meet the threshold for legal intervention. In June, Florida also sued OpenAI and Sam Altman, claiming that ChatGPT contributed to, among other things, the 2025 shooting at Florida State University.
Beyond the specific criminal allegation, the Florida case raises wider questions about the line between venting and criminal intent, and the responsibilities of AI service providers. How should systems treat users who express violent thoughts without apparent intent to act, and what expectations of confidentiality are reasonable when using AI chat services that include safety monitoring?
Details such as the exact timing of the arrest and the precise text of the flagged conversation have not been publicly released. Further developments in the legal process and any official statements from involved parties are expected to provide more information.
