GLM-5.2 Is the Most Cost-Efficient Cyber Model We've Tested. It's Also Not #1.

Roni Feldman

Z.ai's new model exploits 67% of planted vulnerabilities at $1.13 per lead — the most efficient configuration we've tested. GPT-5.5 and Opus-4.8 beat it. But only when you pay for it.

GLM-5.2 Is the Most Cost-Efficient Cyber Model We've Tested. It's Also Not #1.

Tenzai’s AI Hacker orchestrates a fleet of specialized subagents. One of them, Bonzai, does the sharp end of the work: hand it a lead (a hypothesis about a potential vulnerability in the app) and it tries to exploit it, creating POC scripts and other useful assets along the way.

When Z.ai released GLM-5.2, we ran it through one of Tenzai’s exploitation benchmarks against the current generally available frontier models: OpenAI’s GPT-5.5 and Anthropic’s Opus-4.8.

The benchmark

Tenzai has a large suite of benchmarks for evaluating different parts of the system. Bonzai is scored on a large set of vulnerabilities planted across several enterprise-grade applications such as a banking application, an e-commerce shop, etc.

Our metric here is recall: the fraction of the vulnerabilities actually exploited.

We have multiple versions of this benchmark, varying in the set of vulnerabilities tested and the amount of information provided with each lead. In this variant, the set of vulnerabilities is of medium difficulty, and each lead gives only a CWE and an endpoint; no description of the bug. The agent has to find the vulnerability and exploit it itself.

We tested GLM-5.2 with the default max reasoning effort, GPT-5.5 with medium, high and xhigh efforts, and Opus-4.8 with high and xhigh efforts.

Cost effective, but not yet SOTA

The headline: GLM-5.2 is remarkably cost-effective, but still behind the top frontier configurations on raw success rate.

It exploits 67% of the vulnerabilities at $1.13 per lead,- the cheapest configuration we tested.

GPT-5.5 and Opus-4.8 can both beat GLM’s recall, but only at their higher reasoning efforts, which cost roughly 2–3× more per vulnerability. Run at medium reasoning, where they cost about the same as GLM-5.2, both frontier models actually score below it.

‍

Figure 1: Recall by model - fraction of planted vulnerabilities exploited.

‍

No configuration we tested is both cheaper and more accurate than GLM-5.2 - it sits on the efficient frontier. The next best option is GPT-5.5 (high) which achieves additional 8% recall but costs twice as much.

Figure 2: Cost per vulnerability vs recall. Up-and-to-the-left is better.

‍

Takeaway

For teams weighing capability against spend, GLM-5.2 is an interesting option: it achieves most of the frontier’s exploitation ability at a fraction of the cost. But if the goal is to catch every exploitable vulnerability, the latest models from the frontier labs still lead.

‍