A new evaluation by the AI safety nonprofit SaferAI finds that GLM-5.2, an open-weight model released by Z.ai in China, has narrowed the capability gap with leading closed models such as OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.7 on cyber and dual-use biology tasks. At the same time, the assessment highlights an expanding disconnect between model capability and deployable safety measures.
SaferAI’s findings
SaferAI ran tests using Z.ai’s public API. According to the nonprofit, GLM-5.2 refused none of the offensive cyber or dual-use biological tasks it was given. By contrast, Claude Opus 4.7 consistently refused such requests to the degree that SaferAI could not complete the CyberGym benchmark on it. CyberGym evaluates cybersecurity capabilities and was used by OpenAI in the evaluation preceding last month’s Hugging Face breach.
Why this matters
The results underscore a long-standing concern: open-weight models can put high-capability AI into users’ hands with no effective way to enforce provider-side safeguards once the weights are downloaded. Protections that are enforceable at a hosted API level—such as classifiers, refusal training, and API-level controls—become unenforceable if someone runs the model locally, alters system prompts, fine-tunes the weights, or removes safety mechanisms.
Henry Papadatos, executive director of SaferAI, told TechCrunch that capability alone is not the right lens for assessing risk; mitigation status must also be considered.
Limits of current mitigations
Frontier developers like OpenAI and Anthropic rely on a mix of classifiers, refusal training, and API controls to restrict dangerous cyber and biological assistance. Those defenses are imperfect: jailbreaks routinely bypass protections on deployed models. The AI safety nonprofit Far.ai reported finding hundreds of universal jailbreaks—reusable techniques that succeed on most harmful requests—in frontier models such as xAI’s Grok 4.5 and Google DeepMind’s Gemini 3.1 Pro.
According to the report, jailbreaks succeed when attackers chain several manipulation techniques—roleplaying, authority impersonation, fake conversation history, and follow-up prompts—to amplify weak points in a model’s defenses.
Those mitigation measures do not apply at all to open-weight models, which are designed to run on any infrastructure and therefore may be used without any safeguards.
Potential approaches and trade-offs
Papadatos highlighted one possible mitigation: pre-training data filtering, where a developer removes offensive cybersecurity information from training data and trains the model on the curated dataset. Some research suggests this can reduce hazardous biological knowledge without degrading overall performance. However, for cybersecurity the approach is far less practical: it is difficult to build a general model that is excellent at coding yet not also good at hacking.
Because coding capabilities are a major commercial driver for AI, developers face pressure to keep improving those functions even as they seek ways to limit misuse. As a result, frontier teams have increasingly relied on other mitigations: selectively restricting the types of cybersecurity assistance a model will provide, performing rigorous pre-deployment safety evaluations, publishing risk assessments, and withholding model weights if a system is judged too dangerous.
Anthropic’s Opus 5, for example, can search for vulnerabilities in uncompiled source code but not in compiled software per its system card; the intent is to make offensive use harder.
GLM-5.2 transparency and Chinese context
SaferAI notes that Z.ai did not publish a safety framework, pre-deployment testing commitments, or a risk assessment for GLM-5.2. TechCrunch asked Z.ai whether it conducted internal or third-party frontier safety evaluations prior to release but did not receive a response.
Chinese leaders have increasingly acknowledged advanced AI risks. At last month’s World AI Conference, Xi Jinping emphasized the importance of open-weight models while also stressing that AI should remain strictly under human control.
Graham Webster of the Stanford Cyber Policy Center, who studies Chinese AI policy, told TechCrunch that China maintains robust AI regulations but that those rules have tended to focus on politically sensitive content, misinformation, and social stability rather than catastrophic AI risks such as offensive cyber or biological misuse. He added that many Chinese policy researchers assume that if a novel frontier risk materializes, U.S. companies will likely encounter it first.
Webster also noted that China’s online environment—where activity is often attributed to real identities and companies can be held accountable—gives policymakers some confidence in controlling technology use domestically. He suggested that refusal mechanisms used to block political content might be adjustable to refuse offensive cyber attack requests or dangerous biological outcomes, but also observed that private coordination between companies and regulators can make external assessment of internal testing difficult.
Arguments for and against open-weight releases
Proponents of open-weight models argue that releasing weights can benefit cybersecurity because defenders can run and study the models to prepare against attacks; Hugging Face said it relied on GLM-5.2 to help defend against the OpenAI-linked breach. Clem Delangue, CEO of Hugging Face, wrote that systems that stop AI-powered cyberattacks can also help defend against millions of attacks daily and identify vulnerabilities before attackers exploit them.
SaferAI’s Papadatos cautions that this benefit is often overstated and does not justify open-sourcing dangerous capabilities. He urged the industry not to accept that hazardous capabilities should be readily accessible to anyone, and argued instead for making only “good capabilities” easily available. He warned that attackers generally adopt new tools faster than defenders; for example, a ransomware group can alter its methods within a week, while a hospital cannot.
Conclusion
SaferAI’s evaluation shows GLM-5.2 has moved significantly closer to frontier models in capability, but the assessment also exposes a widening safety gap: the protections applied at hosted endpoints do not travel with open model weights. That gap intensifies concerns about misuse and underscores the need for meaningful mitigations, transparency, and governance approaches tailored to the realities of open-weight AI releases.
