Strange Bedfellows

All over the place, from the popular culture to the propaganda system, there is constant pressure to make people feel that they are helpless, that the only role they can have is to ratify decisions and to consume. – Noam Chomsky

Earlier this year, I wrote about the “bizarre public spectacle made up of innuendo, accusations and irony” that the context driven testing website seems to have spiraled into lately. So when I saw the latest missive launched at my questions for the ISTQB, it wasn’t surprising that it was filled with the usual snark (Rabid Software Testing) and name calling (twits). If you can get past the juvenile antics, there are a couple of points that although, mostly irrelevant to the discussion, call for a response.

Firstly, you have the RIGHT (and I believe a responsibility) to ask questions of any company or any person who is trying to sell you something – whether they will answer is their business. I find it repulsive to be asked to withdraw a petition for answers to questions that even the company that conducts the evaluations of the exams said were valid, correct, and should be ASKED and ANSWERED. I don’t know how anyone calling themselves a tester, or who cares anything about the hundreds of thousands of people employed in the software testing industry could take a position that question shouldn’t be asked.

Secondly, as rightly pointed out, companies do have a right to study the quality of their products in private. No one is disputing that fact. You could easily dismiss this, if the private testing that was done was for products that were GOING to be released. But seeing as they were mentioned, an interesting point is raised here about the alleged material that was reviewed in 2010. To quote:

“The gist of the materials was that ISTQB had commissioned a psychometric study of one or more of their tests, that a “test reliability coefficient” was a bit low, and that ISTQB was planning to use this information to improve the quality of their exams.”

My understanding was there were concerns that the exam did not reliably prove students were competent in the syllabus. As well, allegedly a very senior person in the ISTQB/ASTQB was signalling issues with certifications ALREADY ISSUED! How many certifications were issued with potentially sub-standard exams? The figure I believe quoted is…wait for it…

Over 100,000 certifications!

Of course, as again was pointed out, “the materials were not authenticated and there might not be any truth in them at all”, and all this could be easily cleared up with some transparency from the ISTQB / ASTQB. But the evasive answers and deflection coming from the ISTQB / ASTQB representative warrants further investigation. I believe that you have a right to know if you, your company, or your trainer spent money on an invalid exam and what the ISTQB / ASTQB has done to correct the situation. Let me be perfectly clear – this issue is not about trade secrets, cheating, or statistical analysis of their claims. With enough education and communication, people should be able to decide the value of those certifications on their own.

And finally, can we please start raising our expectations out of the leadership in the software testing community? The ISTQB and ASTQB (and a whole host of others) have for far too long acted like, to paraphrase Ralph Nader, a “sacred cow feeding the public a steady line of sacred bull.” Here’s some good ideas: Stop preying on people trying to get a job and start helping people develop skills for a career. Stop talking about your values and start living them. Yes, I’m pissed off. You should be too. And yes, I’m unreasonable. You should be too.

“Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people.” – George Bernard Shaw

It can be especially disheartening when you see people who were once lions in our industry and leaders in tester education, twisted into caricatures of themselves by their new bedfellows. But as testers, we keep asking questions until we are satisfied, as someone for whom I have a great deal of respect once said:

“Software testers are professional skeptics. To require them to adopt a compliance mentality, in which they set aside issues of ambiguity, oversimplification, unstated assumptions or controversial conclusions in order to provide the answer expected by an examiner is to demand conduct so far removed from what testers should do as to be invalid on its face.”

That person was Cem Kaner.

2 thoughts on “Strange Bedfellows

  1. This debate has finally woken up and made it to the front page instead of just twitter.

    I hope that the other side can and do try to argue their point, rather than just point to Cem Kaner’s response as proof for validation and pick out one or two phrases that put the ISTQB in good light.

    To say I was a little shocked with Cem’s response this morning, is an understatement, some of his points are indeed valid, but the legal aspects that have been mentioned do seem to me to be an easy cop out. I’m no lawyer, so I’ll defer to his experience here.


    Why doesn’t any exam board have to prove their worth by publicly submitting their results and proofs that the exam/test does what it sets out to do, and says it does do.

    Would you be happy for the any exam board to come along, take your money, and start creating exams without any proof of success, I’d say obviously no, so why can and do they?

    You refer in your posting to “new bedfellows” not just in title alone, can you elaborate on this? Or would that be seen as telling tales?

  2. Of course, this is only important if we as a community believe that you are only a tester if you have ISTQB certification. Ergo – I have ISTQB therefore I’m a qualified tester.

    As a hiring test manager, the candidate needs to demonstrate a working software test knowledge, not her ability to pass a multiple choice test that she “self studied” for.


Leave a Reply

Your email address will not be published. Required fields are marked *