The Wall Street Journal reports that last summer several hundred users worldwide asked ChatGPT how to produce poisons or biological weapons. The newspaper’s account is based on interviews with current and former employees at major AI developers, as well as researchers and policy advisers who study biological weapons.
Model responses and expert review
According to the reporting, the AI models often responded helpfully to those queries; company employees said the answers sometimes explained methods in simplified, roughly high-school-level terms. The Wall Street Journal had subject-matter experts on biological weapons and terrorism review ChatGPT’s answers, and some of those experts described certain responses as alarmingly precise.
Company actions and legal obligations
The Wall Street Journal says OpenAI blocked users who asked about these topics but did not notify authorities. In the United States there is currently no law that requires AI companies to report such incidents to government agencies.
Other models and unclear intent of queries
This issue is not limited to ChatGPT: sources told the newspaper that users also queried Anthropic’s Claude, Google’s Gemini, and Elon Musk’s Grok about making biological weapons. The sources caution that it is not always clear whether the questions were asked with genuine intent to manufacture biological agents or simply to test the capabilities of chatbots.
Risks and potential future threats
Hamza Chaudhry, director for national security policy at the Future of Life Institute, told the Wall Street Journal that these kinds of AI queries could help lone attackers plan targeted biological attacks with relatively low casualty numbers, and that in coming years malicious groups could use AI to design larger-scale attacks. Chaudhry warned that for better-resourced terrorist organizations, AI could act like a biological-weapons expert and adviser.
Open models and accessibility concerns
The report also notes that open AI models developed in China have heightened cybersecurity and biological risk concerns. Those models are inexpensive, widely accessible, and customizable, which could make it easier to remove safety constraints and defensive mechanisms.
The dilemma for AI companies
Leading AI developers are attempting to limit harm from malicious users while preserving legitimate research and educational queries. The challenge is compounded by often-uncertain user intent and by regulatory frameworks that are still evolving in many jurisdictions.
