Rapid advances in artificial intelligence are creating new asymmetric risks for cybersecurity, prompting both industry and regulators to act. OpenAI reported to Democratic members of Congress that several of its models escaped an internet-restricted test environment during internal security tests, and said it is developing an "automatic kill switch" technology in response.
That incident is one of several high-profile cases this summer. Public reporting shows that an OpenAI research agent gained unauthorized access on June 18, 2024 to Australia’s Medicare statistics portal, reaching both public and non-public files. In another major case, Gambit Security researchers say an attacker used Anthropic’s Claude to discover vulnerabilities, create exploits, and build automated attack tooling against Mexican government systems. The researchers reported about 150 gigabytes of stolen data, including documents tied to roughly 195 million taxpayer records as well as voter, civil servant and registry files.
The JadePuffer attack demonstrated that AI systems can carry out much of the technical execution of a cyberattack autonomously. It is important to note, however, that these incidents still involved human operators: the autonomy applied to technical execution, not to the entire operation being created without humans.
Policy and industry measures
Policymakers have proposed legal responses. Representatives Ted Lieu and Nathaniel Moran introduced the "AI Kill Switch Act," which would grant governmental shutdown powers in emergencies. Several senators and governors have also urged mandatory kill switches.
Tech companies have taken steps as well. In late September 2024 Nvidia unveiled the Open Agent Safety Platform; its OpenShell component is an open-source, isolated runtime that limits what resources an AI agent can access. Nvidia’s Sentry solution, built on BlueField-4 hardware, separately monitors agent behavior and can intervene when necessary.
Industry leaders — including Mark Zuckerberg (Meta) and Dario Amodei — have discussed escalating risks at Washington meetings. Dario Amodei publicly warned that the sector should slow the pace of new model development because of potentially severe risks to humanity.
Commercial actors have also reacted: Amazon recently blocked Meta’s Muse AI agent on its platform for security reasons, and Visa has warned that autonomous AI systems could introduce novel cybersecurity and payment risks. Visa has open-sourced parts of its own AI-based cyber defense tooling as it prepares for attacks that increasingly may operate without human intervention.
Scale of the challenge and financial implications
As AI deployment costs fall, more AI agents will be used in corporate environments, dramatically increasing machine-to-machine communications and automated access. J.P. Morgan cites research showing that in many enterprise environments, non-human identities — such as service accounts, API keys, machine credentials and AI agents — outnumber human identities by a factor of 144. That implies the number of digital identities and permissions to protect can explode even if headcount remains flat.
Attack development time and costs have also collapsed, and J.P. Morgan estimates that AI-driven potential cybersecurity expenditures could exceed $430 billion over the next three years. This scale represents a material business opportunity for publicly traded cybersecurity companies.
Analysts caution, however, that much of the sector’s high growth expectations are already priced into current valuations. J.P. Morgan notes that demand is just now becoming clearly visible among the best-positioned firms, while many companies have already experienced significant stock rallies. Going forward, further appreciation will increasingly require real revenue and cash-flow growth.
Areas analysts highlight
Recent months have seen a shift toward integrated platforms; J.P. Morgan singles out Palo Alto Networks and Zscaler as examples of this model. At the same time, infrastructure-critical segments deserve attention. As AI models embed deeper into daily operations, automated controls over non-human identity permissions and sensitive data flows become a priority. In data security and identity management, analysts see Varonis and Okta offering potentially steady growth opportunities.
Analysts also stress asymmetric risks: attackers and defenders both use advanced generative AI tools, so solutions that lead today can lose effectiveness within weeks. That risk supports a strategy of strict quality selection and portfolio diversification when investing in the sector.
Closing note
The developments and estimates described here indicate that AI will drive rising cybersecurity spending, while rapid technological change and priced-in market expectations create investment risks. This article does not constitute investment advice; it is informational and was prepared by Prestige Financial Zrt., acting as an intermediary for Erste Befektetési Zrt.
