“To be in opposition is not to be a nihilist. And there is no decent or charted way of making a living at it. It is something you are, and not something you do.” – Christopher Hitchens, Letters to a Young Contrarian, 2001

You’re not wrong, everything is getting worse…

Sigh…I’ve had my finger on the “publish” button for different versions of my initial thoughts on AI in testing for far too long, but this great post by Maaike Brinkhof finally pushed me to get it out there. This first one is context setting for a series I’m writing about AI, incorporating it into software testing, the dangers and opportunities, and what I see as a general failure of “duty of care” by our industry.

So much great writing and research has been done by Michael Bolton, Harry Collins, Dagmar Monett, (and too many others to mention) I can’t keep up with it all, so getting into the details of the technology and all the unverifiable claims being made doesn’t seem to have a point in my adding to that pile. Instead, I want to focus on the business risks associated with AI in testing, logical approaches as organisations go headfirst into the mist, and what I think are common sense ways to work with these new tools.

In my opinion, the pace at which people are trying to incorporate “artificial intelligence” into software testing is well past ludicrous speed and quickly approaching “plaid”. Even the halcyon days when the odd tool fetish was the only thing compromising testing seem mild compared to today’s rabid AI bell ringing of every vendor, tool jockey, supplier, and LinkedIn lunatic. This foaming at the mouth is despite the immense ethical and technological challenges AI presents for the testing business and an IT industry that is so far out in front of regulation, they don’t even seem to be playing the same sport.

The mask has been off the major tech players for some time as they disband their ethics teams and software testing vendors rush to build “solutions looking for problems” – all by design. I think it is apparent now more than ever in my 20+ years of working in testing that our business has been completely highjacked by people who have never had to answer for or LIVE with the consequences of their bad quality decisions.

I’ve been saying for years that despite all the “advances” in software testing technology, we and our machines aren’t getting better at testing – we are getting dumber and our expectations eroded by bad software and processes. I’ve long believed in the principle of “Enshittification”, but I think Edward Zitron put it better:  

“We, as people, have been trained to accept a kind of digital transience — an inherent knowledge that things will change at random, that the changes may suck, and that we will just have to accept them because that’s how the computer works…”

Well, I think with all this AI sand getting chucked into our eyes, our industry and the general public are about to get their comeuppance after decades of caretaking technology management (no moral hazard in what we unleash on customers), and the vast majority of people I speak to in this business are not prepared for it.

Leopard, meet face…

As someone who as worked in an industry that’s been undervalued, commoditised, downsized, automated, and abused by every wave of “do we finally get to fire all these testers” technology, I would be lying if I said the “they took our jobs” noise coming from some of the development community hasn’t put the occasional grin on my face. But I would take a bucket of salt with the stories that Meta, Salesforce, or whoever is actually trying to replace programmers with AI, as IME, all the hype is being used as an opportunity to cut costs, repeal remote working policies, and trim operational management more than some new wave of engineering resources. But right on trend in the testing world, what we’re presenting as “testing” and “AI” is not that valuable but can also accelerate bad outcomes and increase risk to your business.

It’s a little hard to keep up with the shifting sands of AI in testing right now as vendors frantically search for a viable product to sell, but I don’t see prompt engineering for test automation or anything else in the market right now as anything other than the lowest and least valuable fruit. Companies have been trying to get rid of humans in testing for decades, but outsourcing critical thinking to LLMs is about as closed a loop as you can get and run big transitional risks.

Most of the clients I speak to these days haven’t incorporated an AI element to their test approach and frankly, the distorted signal coming from our business hasn’t helped. What I’m hearing from clients are big concerns around data privacy and security, transparency on models and good evidence, and the ethical issues of using AI in testing.

That gives me some hope that people are thinking critically about AI in testing as with the recent roll back or cancellation of DEI programs, the least represented communities are traditionally the most harmed and need our highest duty of care.

Inside every cynical person is a disappointed idealist…

“If the rise of an all-powerful artificial intelligence is inevitable, well it stands to reason that when they take power, our digital overlords will punish those of us who did not help them get there. Ergo, I would like to be a helpful idiot. Like yourself.” – B Gilfoyle

I’ve spent a good part of my public career in testing talking about risk, how to communicate it to leadership, and what good testing contributes to that process in helping identify threats to your business. So I’m not here to tell you “no”, I’m here to talk about how we manage through the current mania and what I think are the big rocks we need to move to get there and as an inverse –Carlin, maybe find some idealism in my cynicism.

Testing is more than test cases, So as I patiently wait for the regulars to call me a Luddite (sigh), or I’ve only worked with juniors (womp, womp), or that my opinion is purely self-interest in keeping my “manual testing” job (the lack of creativity from some of these folks is astonishing), I’ll move on to the next one in this series: The Great Liberation Part II: The Search for the Real In Software Quality and Testing

Enjoy!