In our standard cynical mix of technologists first world willful ignorance, moral relativism, and “bought and paid for” integrity, there’s more high-profile resignations in the world of AI safety with a “slow down” called for by the biggest labs at the frontier.

In a recent statement, the head of Anthropic, Dario Amodei said this about the need for more testing:

“Testing and evaluation of AI models becomes more difficult as they increase in capabilities. More intelligent models are more capable of deceiving tests, and thus may appear aligned while having serious problems that go undetected. Building up a much broader and more ingenious stable of evaluations, along with interpretability analysis to cross-check them, would be hugely valuable, and a lot of progress could be made on this in 1-2 years.“

Aside from the anthropomorphic framing which side steps responsibility for the deployment of these tools, he makes it seem like this is just a capacity problem when the real issue is that these systems operate inside socio-technical environments where it’s impossible to predict all the modes of failure and the primary approach to testing has been throwing more AI the problem.

This is an old mistake dressed up as an AI safety insight: assuming that if testing is hard, the answer is simply more testing. It isn’t. If models can “deceive” evaluations, then the problem is not a shortage of clever tests, it’s that you cannot know your evaluations are exposing the behavior that matters in the first place. And the idea that this can be substantially “solved” in 1 – 2 years is not ambitious, it’s detached from what we already know about testing.

For testers, this is another case study in why confidence is not the mission of testing: they’ve worked out why testing cannot give them the confidence they want, and their proposed solution is more testing designed with the same assumptions that created the confidence problem in the first place.

Further to that, everything you’re reading from the ad agencies we call a press these days is AI marketing masquerading as doomerism, and the software testing industry better snap out of it and recognize the completely cynical grift that’s being foisted on our community. So let’s get a couple of things straight:

No. Software can’t check its own quality.

No. More tests does not mean better testing.

No. “Complete test coverage” is not a meaningful claim.

No. Engineering confidence is not a discipline.

And no, a human clicking “approve” is not a meaningful control.

Working in testing comes with a professional responsibility to tell the truth. If you’re not up for the job, on behalf of everyone who relies on our business for that truth, you should see yourself out.