A while ago, I wrote a series of posts on how testing can support your business through a “transformation”, either through disruption, optimization, or by managing expectations around risk and what testing can do to help navigate digital transformation. As GenAI has stampeded into every aspect of technology delivery and innovation, I thought it only appropriate to add another entry into my “Test as Transformation” series.

I’ve always contended delivering technology value to your business will either be constrained or accelerated by your approach to testing, and adding AI into that risk is like injecting it with steroids. By layering black boxes into everything from customer service to decision support, testing has rapidly become one of the most important sources of business intelligence a company has and a true market differentiator.

Testing has always been about information – learning things about systems we didn’t know and confirming things we thought we did, but there are new risks AI presents for your business I’d like to highlight that testing should address.

Regulatory and Oversight Risk

Testing (when done right) should bring insights into risk, anticipate issues, and surface patterns which is even more valuable in an AI supercharged competitive landscape. As the AI regulatory environment continues to evolve due to technology changes and what seems like capitulation to big AI firms, we need to adapt to this “emergent” regulation and yet to be revealed enforcement.

Testing has always been at the heart of regulatory oversight, and I’ve spent a career in banking writing responses to regulators and answering internal and external audit questions. We need to be prepared for new AI oversight and evidence for:

• Algorithm transparency
• Automated decision-making controls
• Governance for training data and model drift
• Audit trails for traceability on workflows

Having the wrong strategy for testing AI enabled systems can introduce risk to your business through biased, unsafe, or inaccurate outputs that may violate new or existing laws as well as industry standards like the EU AI act, GDPR, or new product liability laws. I believe we are shortly going to be litigating harmful outcomes, privacy breaches, and misleading claims of which all could have been triggered by audits of weak governance and inadequate risk management.

Test Optimization Risk

The claims of test optimization and efficiency gains through automation are nothing new to the testing industry. Shallow automation-based checks have been the ROI currency of “modern” testing but just lead to overconfidence and very frequently larger systemic failures. Transformative testing must guard against the usual set of automated business risks and now address AI enablement like:

• AI assisted test generation missing coverage faster and broader
• AI powered code analysis injecting systemic patterns of fragility
• ML based anomaly detection clouding operational visibility
• AI driven customer behavior insights creating misleading test conditions

Managing the delta between AI-powered checks and human-led tests addresses the dangerous belief that AI automation equals AI quality. And just like traditional digital transformation, culture and incentives drive behavior that increase the threats to value in your business. Test automation failed to deliver on the core of its promised ROI, and AI enabled testing frameworks only seem to double down on familiar tropes:

• AI will remove all experiential testing
• AI will find and remediate all defects
• AI can replace testing
• AI systems “learn on their own” without oversight

Testing now needs to validate not only functionality but also non-deterministic behavior of AI systems to ensure fairness, transparency, bias mitigation, and regulatory compliance and cannot rely on traditional processes.

If testing is to survive an AI transformation, it means helping redefine how the organization operates, delivers value, and competes. AI can expand that definition by introducing new automation capabilities, intelligent decision-making, and previously impossible efficiencies. AI magnifies both the opportunity and risks associated with the mission of testing but must be paired with human judgment, domain expertise, and systems thinking.

We’re at the start of a global AI transformation and testing has to move from a supporting activity into a strategic information engine to keep your business competitive.