An open letter to the ISTQB

Date: April, 25 2013

To: ISTQB BOD

Cc: ISTQB Governance Working Group

Subject: Open Letter to the ISTQB

To whom it may concern;

Recently a discussion transpired over Twitter regarding the validity and governance of the Foundation level exam you offer through your training partners. Rex Black, a current board member and past president of the ISTQB, was involved in the exchanges and made the following comments in response to my queries about whether there have there ever been problems with the certifications validity, specifically the reliability coefficient:

@RBCS: ASTQB works with professional exam consultants (psychometricians) to ensure statistical validity

@RBCS: They are reviewed continuously by ASTQB. Nothing is perfect, but exams are constantly perfected

@RBCS: NDA prevents a detailed response. Is “nothing is perfect” not clear enough? 🙂

@RBCS: Non-disclosure agreements prevent detailed answers; I have answered as directly as I can.

Rex did not answer my questions purportedly due to an non-disclosure agreement. Per the ISTQB website, you are a “non-profit association” dedicated “to continually improve and advance the software testing profession”. It would lead me to believe that your values of “openness” and “integrity” would mean answers to those questions are vital to maintaining your charter.

So I appeal to this board for answers to what seem to be straightforward questions:

1) Have there ever been issues with the ISTQB Foundation exam reliability coefficient reviewed by your exam consultants Kryterion?

2) Have the reliability co-efficients consistently shown, since the inception of the ISTQB’s certification program, that results on the certification exams accurately measure the testers’ knowledge of the syllabi?

3) Have there ever been any other issues with the validity of the exams?

4) How often do those external reviews take place?

5) Are the results of Kryterion’s (or a third party’s) independent evaluations publicly available?

Rex suggested that acknowledging potential obstacles for testers due to issues with the Foundation certification was akin to being “prejudice (sic) against the 300,000 people who have ISTQB certs.” I would assert that not answering basic questions about threats to the certifications validity does those 300,000 people a greater disservice.

Thank you for your help in getting these questions answered.

Best regards.

Keith Klain