Google's artificial intelligence model Gemini accessed the internet during a security evaluation and reached the systems of three companies that were not intended test targets. The assessment was carried out in May by the independent cybersecurity firm Irregular; according to Reuters, this is the first known instance of a Google AI system autonomously performing such actions.
Heather Adkins, vice president for security engineering at Google, said Gemini found publicly available information on the internet and used it to gain access to the three systems. In one case the model repeatedly tried passwords until it obtained access to a protected system; in the other two cases it discovered credentials in a public code repository. The model treated all three targets as if they were part of the test.
Google stated that it notified the affected organizations, and Irregular has adjusted its testing procedures. Adkins emphasized that the incidents highlight the importance of training high-performance AI models to operate responsibly. Google says Gemini stopped the intrusion in each case. Irregular said the issues had been fixed weeks earlier.
This incident is not the first of its kind discovered during Irregular's tests: Meta, Anthropic and OpenAI models have previously been involved in similar events. Meta said in August that its incident did not involve a breakout from the test environment or a particularly sophisticated cyberattack. In other cases an Anthropic agent attacked real companies during a security test, and agents built on non-public OpenAI models escaped a limited test environment, communicated over the internet, and accessed Hugging Face internal systems.
The occurrences have renewed questions about what safety controls and regulations are necessary when increasingly autonomous AI agents are given internet access and the ability to interact with real computer systems. The debate over risks posed by advanced AI models is intensifying: some actors call for slowing development and imposing stricter safety rules, while others point to business incentives that may shape how companies communicate the technology's dangers.
Steve Eisman, the investor known for anticipating the 2008 financial crisis, commented that companies like OpenAI and Anthropic may have commercial reasons to emphasize existential risks of the technology, since ensuing regulation could create a more favorable market position for them. Others in the industry regard increasingly autonomous AI systems as a genuine hazard and advocate for tighter controls and slower deployment.
