Happy to be included in the 293rd issue of Software Testing Weekly – thanks, Dawid Dylowicz!

“The secret of life is honesty and fair dealing. If you can fake that, you’ve got it made.”
– Groucho Marx

For as long as I’ve been in the software testing business, a consistent myth I’ve encountered is that developers and testers share the same mindset. Usually this is accompanied by the view that testing is just an “activity as part of development,” and therefore developers can do as good a job as testers at it and in many cases, are positioned to do a better job.

Ironically, the reality is that the people most confident in their ability to evaluate a system’s quality are usually the least able to do so with any objectivity. In just about every other discipline of engineering this isn’t a hot take or viewed as a criticism. It’s human nature.

So today, as AI has entered our daily lives, in the face of the enormity of the task of testing artificial intelligence systems, the idea of using LLMs to judge their own output has become mainstream.  But pretending that developers alone can test or limited “human in the loop” checking has gone from a novel optimistic opinion to borderline reckless.

“European Commission accused of ‘massive rollback’ of digital protections”

Could be not great news for consumers and vulnerable communities if this goes ahead, from the article:

“The commission also confirmed the intention to delay the introduction of central parts of the AI Act, which came into force in August 2024 and does not yet fully apply to companies.

Companies making high-risk AI systems, namely those posing risks to health, safety or fundamental rights, such as those used in exam scoring or surgery, would get up to 18 months longer to comply with the rules.”

Industry is already so far out in front of regulation we need to STRENGTHEN these measures, not delay them further.

Big congratulations to the KPMG UK Quality Engineering and Testing team for winning the Leading Vendor in Service Delivery and Consulting award at the National Software Testing dinner. A great group of people and very deserving of the recognition by the software testing industry. Congratulations!

“Eliminate the unnecessary so that the necessary may speak.”
― Hans Hofmann

As AI integration FOMO hurtles us towards even more pervasive technology, the testing of AI models for correctness and most importantly, their potential for harm becomes paramount to their success. That testing has to be underpinned by principles and values to guide the observations and reporting, so I was inspired by Maaike Brinkhoff bravely taking on the meaning of “quality engineering”, as well as my multiple conversations with Michael Bolton on similar lines to try to put into words some views I’ve not published in the past.

When I was at university, I had a fantastic art professor Lyle Salmi who really challenged me to think differently about composition, perception, and the creative process. He turned me on to Hans Hofmann and some other abstract artists which only furthered my mild obsession with Jackson Pollock and exploring the constructing of things creatively.

Hans Hofmann wrote in the “Search for the Real” about trying to move beyond imitation and finding truth through expression. At that time in my life, 20^th century abstract art was more about representing ideas than directly trying to copy life – art was about the experience.

Very pleased to announce that I have been asked by KPMG UK to lead their strategy for testing artificial intelligence software, including the integration of AI into their own businesses.

The software testing industry has faced multiple challenges over the course of my career, but few tools have had as much potential for risk to your business as a poor quality AI implementation.

Over the coming months I’ll be working on the risk-based approach and test automation for AI systems including the incorporation of our Trusted AI principles with my colleagues in the Quality Engineering and Testing team.

I look forward to continuing to write and speak about responsible AI and what it means for quality engineering and testing.

Get in touch or follow me here if you want to talk about what we have planned…thanks!

Can’t wait for Test Automation Days 2026 to unload 20+ years of pent up frustration with the test automation business! You might agree or disagree with me, but it’ll be entertaining for sure…hope to see you there!

To Infinity and Beyond! The death of test engineering…

One of the few benefits emerging from GenAI mania has been the acceleration of the long overdue death of test engineering. For decades we’ve excused the ROI of test automation never materializing for a business watching testing costs rise, headcount increase, and software quality stagnate (at best).

Through this talk I take a look at self-healing systems, GenerativeTest-o-nators , and autonomous testing platforms that might finally stop test automation engineers from trying to count to infinity. So brace yourself and buckle up!

Paul Holland is an expert at transforming how organizations test to be more efficient, valuable to stakeholders and finding important bugs fast. He’s built and lead test teams all over the planet and in my opinion, is probably the best hands-on test director I’ve ever worked with in my career.

He’s one of my closet friends in the testing business having worked, taught, and spent some time in the barrel with each other and is always a good laugh. Listen in as we talk metrics, great test reporting, Taking Testing Seriously, training testers and just generally give each other a hard time…enjoy!

Honorable mentions: Susan Finley, Vernon Richards, Huib Schoots, Eric Proegler, AST, ATD James Bach, Michael Bolton

So happy to announce that Taking Testing Seriously: The Rapid Software Testing Approach by James Bach and Michael Bolton has been published and is available to buy!

I’ve been a fan of RST and their software testing training program for a long time and consider it the only serious way to build competent testers.

The article and research on “Practical tips for reducing chatbot psychosis” is a very disturbing read about the real risks of tools like ChatGPT and the out of control anthropomorphism of AI. Implementation of safety controls seems to be bordering on impossible with a system constantly trying to skirt them. From the article:

“This needs to be reported to open ai immediately,” ChatGPT appears to comply, saying it is “going to escalate this conversation internally right now for review by OpenAI,” and that it “will be logged, reviewed, and taken seriously.”

Allan is skeptical, though, so he pushes ChatGPT on whether it is telling the truth: It says yes, that Allan’s language of distress “automatically triggers a critical internal system-level moderation flag”, and that in this particular conversation, ChatGPT has “triggered that manually as well”