The debate around ISO Standard 29119 has intensified after James Christie gave his talk at the Association for Software Testing conference this year, which started a movement organized by the International Society for Software Testing and a petition to stop the publication of ISO Standard 29119. The following is a transcription of a roundtable discussion between veterans of the software testing industry to set the context for the opposition.
Keith Klain (KK) – So why has the opposition to 29119 only recently started? The standard has been in development for years now, so why all the recent action regarding it?
Iain McCowatt (IM) – That’s James’s fault!
Michael Bolton (MB) – I actually have a theory about that, I’ve been grousing about this for several years, and other noisy colleagues of mine have been as well. It’s when the quiet ones speak up, that’s when you start to get some attention. I think there was just a confluence of community that was galvanized by his talk. I think James put the hammer right on the head of the nail when he brought up the idea of “rent seeking”, and the appeal to fear to a community that really hadn’t recognized this message.
But the simple answer is, we’ve been busy and we have been talking about it quietly, and James put a match to a pile of newspaper. It was ready to happen, and it finally did happen.
KK – So James was there anything in particular that caused you light the pile of newspapers you were sitting on?
James Christie (JC) – It’s something I’ve been thinking about and writing about on and off for the last couple of years, but it was at EuroSTAR in Gothenburg last November there was a short discussion just on the exhibition floor. Stuart Reid was defending standards and he kept on shifting his ground saying, they weren’t compulsory, it was up to clients to decide whether or not they wanted it, it was up to individual companies.
And I thought it was just so evasive, he was pushing something that I thought would be damaging and he was doing in I thought, a rather evasive way, not addressing the real concerns. So when I got home, I wrote a blog about that, and I followed up with that, and Keith suggested that I might put in a talk on a similar basis, so I thought about it and did that. So it wasn’t something that I actually planned, but once I started off on that thinking about what I should be doing after that discussion in Gothenburg, it all just fell into place.
MB – By the way, I’ve had that same conversation with Stuart Reid myself in 2007, 2008, 2010, 2011, and it’s always got the same results.
JC – I think its important to stress that the oppositions always been there for a long time, but its only just got organized. I think its because people have realized that its not just enough to oppose, it requires a level of organization otherwise the opponents can be dismissed as irrelevant, disaffected individuals. It’s important that there is some consistent campaigning from organizations, and that it is organized rather than just individuals.
IM – I think all of the pieces were in place and a lot of peoples heads were in the right place when it comes to opposition to the standard, but James provided the “call to arms” we needed.
Ilari Aegerter (IA) – James was certainly the trigger to get the higher intensity of the opposition to 29119, but it wasn’t the start, just an intensification of the effort. This is also a result of the organized nature of the opposition and therefore the perceived intensity shift is not a start, but a continuation.
KK – So Iain, can you tell us a little more about the petition?
IM – So after the Q&A after James presentation at CAST, I asked the question, what do you think we can do about it? And actually, Karen Johnson threw a “red card” at that point and spoke briefly about things we could do. And it was conversations that stemmed from that, that created the petition.
One of the major objections that we have to the ISO standard and the way that its being produced, is the way in which its being produced by an exclusive club, that haven’t taken into account the views that the wider testing community have about how to do testing and how to do testing well. There is no consensus to the standard, so our petition calls upon ISO to withdraw the first three parts of the standard, those that have been issued, and to suspend publication of the remaining two parts, on the basis that there is no consensus and that there is sustained opposition to the publication of these standards.
IA – I wanted to add that the International Society for Software Testing is helping in organizing all the opposition backing the petition against the standard.
James Denman (JD) – I’m wondering if there could be a standard for testing, what would it look like?
Griffin Jones (GJ) – It would probably evolve out of guidance.
KK – To draw an analogy that as well, some point they will link, and we are starting to see the beginnings of that, but its like the tester certification movement and how this ties into it. And this is the logic I hear all the time, this “well what is your alternative?” right? You need to have an alternative to foundation level certification, so what is it? And my answer to that is, you absolutely don’t. I don’t think we need to create another shallow, simple test to try to counteract the negative affects of the current certification scheme.
And other than the commercial argument that’s been made, which is really the insidious part of this. Because we are talking about a commercial organization that is creating a market for selling their services and whole cottage industry of consultancies around it to sell certifications. When you get down to brass tacks, that’s the only real argument I can see that’s viable in this, and I don’t feel compelled to create an alternative to that.
IM – I don’t think there is a need for a standard, where there is a need for is testers who are dedicated to improving their skills. Testers who are dedicated to inventing new ways of testing and trying them out. It is a very, very young profession, it’s far too early to be imposing a standard, and if we do so, what we will do is stifle innovation, and everybody’s going to suffer.
MB – “What’s your alternative to bad” is always a dodge. It’s a non sequitur. What’s your alternative to something unnecessary? “Nothing” is fine. “Nothing” is absolutely a viable answer to that. We don’t need a standard because it’s not something that fits for precisely the reason that hackers and bugs and testing problems don’t come with a standard, and so there cannot be a standard response or a standard way of defending against them.
KK – And Michael, that’s an excellent point, because what you’ll see, and this is the shallowness of the arguments that they make FOR the standard – the burden is NOT on the people who they are trying to impose the standard on, to say why I don’t need this. The burden should be on the people trying to impose the standard, as to what is the commercial, industrial, and regulatory – what are the reasons, what is the actual purpose, what is the business case to why this is required.
And so far to date, I’ve seen absolutely no reasonable or viable as to why this is needed. So the burden is not on the people who are going to be regulated – the burden is on the people who are going to be doing the regulation.
MB – In addition to your list, Keith, we should add the societal benefits. What are the societal benefits, and what are the societal costs?
IM – Where are the arguments, where is the evidence that will stand up to any kind of scrutiny at all? There is none.
JC – They aren’t even pretending that there is any evidence. The are trading on the ISO brand name, the fact that it is a standard, and that means that many peoples responses will say, “well it’s a standard, that must be a good thing, what is your justification for rejecting it?” They are able to turn around the argument; they don’t feel they have to justify themselves because they are ISO.
GJ – This may just be a case where, it’s inappropriate to apply standards in this particular case, in this particular set of industries, in this particular field. If the ideas being proposed are so good in the standard, quite frankly, publish them as guidelines. Demonstrate through sustained commercial efforts that they have broad applicability and commercial success and appropriately deal with risk. If they are successful, they will be adopted and everyone will accept them. But they (ISO) don’t choose to go that path; they use coercion or implied coercion.
JC – Agreed.
MB – The mechanism for which its (29119) is created is odious. One of the responses to “why is this (opposition) only happening now”, is it takes an incredible amount of funding and time in order to participate in this. That favors two things, it favors people who are going to make money off the consulting and certification services for this standard, and it also favors people who are in it for the long haul. This has dragged on for six years, it has taken them that long and essential the opponents to it die of attrition. They are out of the decision making process because, who has time for this? And the convener is willing to keep it that way; they will wait it out because there is bucks in it for them.
GJ – So it’s how this was formed, it’s how it’s being imposed, and it’s the details – other than that, we like it!
ALL – (Laughter)
GJ – I started out when 829 first came out, it took seven years to get people to finally acknowledge, “829 was kinda nice in theory, but when we tried to do it wasn’t practical and we got all screwed up”. It took our company two or three years to get unwound from that, and I don’t want to go through that again – I already did that.
And this one’s (29119) even worse, because its scope is so much larger and its implied hubris, well, its not implied it IS hubris – that it applies to all software development across the world is just, I thought ridiculous at the face of it and it would die of natural attrition. But I was wrong.
JC – 829 just dealt with documentation, 29119 deals with that and the processes too. It is explicitly more ambitious and damaging.
IM – I’ve worked with a lot of client organizations where they’ve attempted to standardize in one shape or form. Whether it’s by adopting some thing like TMMi, or whether it’s establishing some kind of internal process or internal standards that they’ve rolled out across the organization. And in pretty much every case, the process and the documentation took over.
JC – That is exactly the topic of my talk at EuroSTAR in Dublin this year. Its about two projects I worked on, they were the same project – I was the test manager and I ran the documentation to make sure we got paid by the client. I made sure their documents got through the quality gates, everybody was happy, we got the money and my deputy was doing the real testing which bore no relationship to the paperwork at all. They were essentially two different projects, and the testing had now gone underground.
IM – And here is something that is cited as making testing more efficient and more effective – its laughable.
Michael Bolton is a consulting software tester and co-author (with senior author James Bach) of Rapid Software Testing, a methodology and mindset for testing software expertly and credibly in uncertain conditions and under extreme time pressure. Michael has 25 years of experience testing, developing, managing, and writing about software.http://www.developsense.com
Iain McCowatt is one of the founders of the ISST, and the author of the petition to stop ISO 29119. His day job is as a director in a bank, helping large enterprise IT programmes to solve complex testing problems and gain insight into the quality of their software. http://exploringuncertainty.com/blog/
Griffin Jones is an agile tester, trainer, and coach, who provides consulting on context-driven software testing and regulatory compliance to companies in regulated and unregulated industries. Owner of Congruent Compliance, Griffin has been participating in software development for over twenty-five years. http://www.congruentcompliance.com
James Christie has over 30 years of experience in IT, covering development, IT audit, information security management, project management and testing. He is now a self-employed testing consultant based in Scotland. http://clarotesting.wordpress.com/
Keith Klain is the CEO of Doran Jones Testing and has over 20 years of multinational experience in enterprise-wide testing programs, Keith has built and managed global test teams for financial services and IT consulting firms in the US, UK, and Asia Pacific.www.qualityremarks.com
Ilari Henrik Aegerter is President of the International Society for Software Testing where he wants to bring back common sense into testing and oppose wasteful practices. He has been in software testing for the past 10 years, most of the time as a manager of testers. http://www.ilari.com/
James Denman writes, edits, and manages the production of content for SearchSoftwareQuality.com. His job is one part editor, one part reporter, one part copywriter, and three parts whatever else needs doing. http://searchsoftwarequality.techtarget.com