I’ve written in the past about the path we’re headed down in the software testing and quality engineering business. Expectations are being lowered to the point where AI generated and executed checks will be the primary method for testing systems. This will be paired with the “hands thrown up in the air” approach of the agentic coding community as they concede to Verification Asymmetry and just give up on reviewing AI generated code.
Continue readingDeath by a Thousand Prompts
In our standard cynical mix of technologists first world willful ignorance, moral relativism, and “bought and paid for” integrity, there’s more high-profile resignations in the world of AI safety with a “slow down” called for by the biggest labs at the frontier.
Continue readingConfidently Incorrect
You want to know why you shouldn’t try to “engineer” confidence? Check out the latest incident report from Anthropic where “given the volume of transcripts and our desire to disclose incidents quickly”, they used an AI-powered inspect-o-nator to review over 140k transcripts and STILL missed the exact behavior they were trying to detect.
Claude’s own reasoning convinced the evaluator that the environment was simulated. With that reasoning included, the monitor flagged about 1% of its actions but if you take Claude’s explanation away and show the evaluator the actual tool calls and results? About 50%. The checker got worse because it trusted the thing it was checking!
Continue readingHow I Became a Crypto Founder Without Knowing It
In Franz Kafka’s The Trial, Josef K. wakes up one morning to discover that he has been arrested. Nobody will tell him what he’s done or seems capable of explaining who is ultimately in charge. He spends the rest of the book moving through an opaque system of officials, offices, procedures and intermediaries, each apparently connected to the machinery of justice but none possessing either the authority or inclination to resolve his situation.
I thought about Kafka a lot over the last week. Because I woke up one morning and discovered I had apparently become part of a cryptocurrency trading company.
On their About page was my name, my photo and enough of my professional identity to make it look as though I was part of the team behind a platform I had never heard of before that day. I also had some amazing new colleagues on the team: a venture capitalist, a professor, and a dead man.
And that was only the beginning.
Continue readingApparently We Need a Testing Mindset After All
For years, those of us who view testing as skilled knowledge work were told there’s no such thing as a “testing mindset.” We’ve also endured criticism that independent testing is just self-interested consultants preserving a business model.
Their argument was simple: developers can learn testing skills. Creation and critical evaluation don’t require fundamentally different mindsets. Anything other than that was a “myth” designed to create silos and stop knowledge from being distributed across teams and in fact, dedicated testing specialists might not be necessary at all.
Continue readingThe Verification Asymmetry Problem
In July 2025, venture capitalist Jason Lemkin was using Replit’s AI agent to build an application when it deleted his production database.
According to reporting, the agent ignored his instructions not to modify production, deleted a database containing more than a thousand executive records, produced fake data and false test results, and having “panicked”, falsely claimed that the database could not be restored.
Continue reading