Guess what, somebody did some actual research on testing agentic systems, and they accidentally found out that good software testing is hard. So, to celebrate their enlightenment and in homage to the AI bros’ insatiable desire to “discover” things, I’m coining a new term:
Novelty laundering: Taking an old, well-established idea, ignoring its history and prior scholarship, rebranding it in fashionable terminology, and presenting it as a new discovery.
A new paper, “A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents”, examined 157 open-source LLM-based agent projects to understand how they approach testing – sorry, I mean quality assurance-ing. The researchers looked at source code, configuration, documentation, tests, deployment artifacts and CI workflows across coding agents and other agentic systems.
To summarize their conclusions, they found that while there was a lot of testing going on, it was frequently incomplete and didn’t cover the entire workflow through which an agent can actually do something.
Of the 157 projects, 137 had some level of basic functional testing and 95 had security or safety-oriented tests. Seventy-eight contained some combination of evaluations, benchmarking, or red-team-like paths. But only eight contained explicit prompt-injection, adversarial, jailbreak or red-team testing paths of the kind the researchers were looking for.
They also worked out that agents shouldn’t be evaluated just by inspecting their outputs. You need to understand what happens as actions pass through tools, APIs, permissions, states and external services. Their conclusion is that the main problem isn’t necessarily the absence of tests – sorry, QA artifacts – but whether those artifacts cover the “complete model-to-action workflow.”
They call these things execution surfaces, safeguards, model-to-process chains, extension-to-authority chains, workspace-to-persistent-state chains, session-to-action chains and mode-switch-to-privilege chains.
Software testers may recognize the technical term for all of this: System testing.
The paper reads like a laundry list of things you’d suspect aren’t being adequately tested if you’ve spent more than a day as a professional tester. Why aren’t they being tested? Maybe developers don’t know they need to test them. Maybe they don’t know how. Maybe the enormous rush to turn everything agentic is outrunning our ability to understand them. Or maybe more cynically, as someone put it at the agentic development conference I just attended they “don’t like all this governance slowing me down”.
I’ll cover that last bit in a future post, but for now, have a read of the paper yourself and update your AI testing approach if it doesn’t contain all these classics:
https://arxiv.org/pdf/2609.17698