“The scientist has a lot of experience with ignorance and doubt and uncertainty, and this experience is of very great importance, I think. When a scientist doesn’t know the answer to a problem, he is ignorant. When he has a hunch as to what the result is, he is uncertain. And when he is pretty damn sure of what the result is going to be, he is in some doubt.” – Richard Feynman

I recently had the distinct privilege of watching an expert tester at work. I wouldn’t call this person a “test manager” or “test lead”, even though what they were doing would probably be categorized as an activity associated with both of those roles. No, I would give them the honor of calling them an expert tester – someone using all of their knowledge and skills developed through years of practicing their craft. And they weren’t even testing software; they were testing ideas. Testing assumptions. Testing people. Testing themselves. It was a thing of beauty.

In my almost 20 years of working in this field, I’ve met only a handful of people who could have been dropped into this scenario and come out the other side alive. I didn’t have a scrap of paper to give this person and had not had a single discussion with the project team yet. No preparation. No context. Nothing. Just a room full of COOs, quants, and extremely senior business sponsors all looking for someone who could help them. The elegance of approach and poise demonstrated quickly punctured the tension in the room, and they were free to set about their work – asking excellent questions and not stopping until they were satisfied.

What struck me the most after the meeting, was the feeling of excitement and exhilaration from everyone in the room. Foolishly worrying that I had set everyone up for failure, I had forgotten one of the hallmarks of an expert tester: they are comfortable with complexity and ambiguity. They thrive on it. And they don’t need “best practices” or a two-inch thick manual on how to get the job done. Their weapon of choice is their mind.

And there lies the difference between the prevalent “best practice” and insurgent “skilled testing” communities. In my experience, the people who espouse “best practices” are the ones that do the least amount of learning and practicing their craft. A skilled tester is comfortable with ambiguity and does not hide behind it or disguise meaning through use of nonspecific language. People obsessed with “best practices” look towards quantification as a way of improving numerical efficiency, and use words like “smug” to define those who feel “Not everything that you can measure matters, and not everything that matters can be measured.”

This approach is similar to what Nasim Taleb describes as a “ludic fantasy“, or incorrectly applying statistical models where they don’t belong, and I believe leads the “best practices” crowd into state of “survivorship“. In that state, despite a complete lack of evidence and scientific analysis, you want to believe that your strategy and practices were the reason you were successful – simply because you didn’t fail. We are consistently “fooled by randomness” to think that we actually know something, and can apply standards and practices without context or skill and get a better result. Wish thinking is a powerful force.

This has been the prevalent approach to software testing for the last decade or more, and fuels the hordes of consultants and testing vendors, foaming at the mouth to commoditize, package up and volume discount testing “best practices”. But when you start to peel back the layers and get into the details, ambiguity abounds and the models quickly fall apart. The hard work to actually “know something” hasn’t been done. What everyone who understands anything about good testing will tell you, is that it is not the practices themselves that make you successful – it is knowledge of their skillful application.

And that knowledge requires hard work and practice to obtain.

“True ignorance is not the absence of knowledge, but the refusal to acquire it.” – Karl Popper

Special thanks to the awesome folks at The Testing Planet for publishing the following story in their latest issue. Get yourself together and subscribe today!

Leadership in Testing – What Really Matters

I’ve hired lots of testers. I’ve hired some great ones, and some well, not so great ones. Some that exceeded all my expectations for them, and some that I thought were bound for “greatness” and fell short of the mark. Consistently, the one quality that I see distinguishing the ones who reach their full potential from the ones who don’t: leadership. I prefer to think of leaders using the definitional term “guide” when describing them. They play different roles under different contexts, but always guiding the organisation, whether it be a team or an individual towards the goal.

Now, it is a very common mistake to conflate leadership with management. A leader can be a manager as well, but as we all know, being a manager does not mean you are a leader. We’ve all struggled under managers who didn’t have a leadership bone in their body, so to avoid inflicting that terror on my teams, the following are characteristics I am looking for in either hiring or promoting leaders:

1) Honesty – I speak a lot about honesty because it’s so important to leading with integrity. It resonates into every aspect of how others see you, and how you see yourself. People want to know that their leaders are telling them the truth to trust them to act as a co-steward of their career. And that trust is built with a healthy dose of self-refection. Admitting you made mistakes, sharing information, apologizing when you’re wrong – good leaders have no fear of the truth. Honesty is the building block on which you’ll build great teams, and it has to start with its leaders.

2) Communication – All great communicators are not leaders, but all leaders are great communicators. Setting the context for the mission is essential to keep people motivated and aligned with the business, and that means you have to be able to relate goals to tasks. People who tell stories that find common threads in our shared experiences are typically the ones who get the most from their teams. In order to propagate an idea, it must be relatable to something we value ourselves.

3) Humility – History is full of examples of leaders with tremendous egos. In order to even want to be in a leadership position, you must have a healthy sense of self-worth. But I think the best leaders can drive organisational change, not as programmatic coercion, but as Dwight D. Eisenhower called “the art of getting someone else to do something you want done because he wants to do it.” That kind of leadership demands humility. A great tell on whether someone has a humble spirit is if they use “I” and “we” interchangeably when they speak about earlier teams, or give a pat answer when you ask them about their last mistake. I want my teams to take ALL the credit because they are the ones doing all the work!

4) Passion – People look to their leaders to keep their foot upon the accelerator, setting the pace for the organisation or team. Passion is what inspires people, and inspired people can do amazing things. I am extremely fortunate that I love my job. But what exactly is my job? My job is helping organisations and people improve themselves through great software testing. I tell my teams that we are not only responsible for improving testing on our projects, but also in the industry. Nothing less! If you’re not passionate about what you are doing, trust me, no one is going to follow you – regardless of your title.

In my experience the best leaders are honest with themselves and others, can speak in stories that tie things together, approach life with humility and their passion inspires those around them. I’ve failed more than I’ve succeeded in finding leaders, but when I have been successful, they’ve met those marks. Best of luck and happy hunting!

“It is rare for people to be asked the question which puts them squarely in front of themselves” – Arthur Miller, The Crucible

I love scrutiny. I love it so much that I try to constantly surround myself with people who challenge my views. Either by directly asking for critique or by putting my work up to public review, I seek honest perspectives on my work to continually improve. I need scrutiny in the same way crucibles are used in laboratories or for scientific purposes: they withstand high temperatures to remove impurities. Scrutiny burns off impurities in my ideas and actions. It clarifies them. And I set the pace and tone for that scrutiny through how I give feedback to others: straight and to the point.

And guess what drives my desire for scrutiny: insecurity. That probably sounds funny coming from me, and might even sound like a weakness, but I assure you it can be one of your greatest strengths if used properly. My insecurity drives my quest for excellence in myself and in turn, in my teams. I am not confident we are always doing the right thing. I am not convinced we are always pursuing the right strategy for tools, process, and people. And it is exactly because I am not 100 percent secure in all my decisions, that I don’t just “like” scrutiny, I NEED it! In the never-ending pursuit of excellence, scrutiny is my compass.

Far too often I have seen testers get crushed under the weight of their own insecurities. And coupled with a fear of failure, that can have a paralyzing effect on either a person or team. Face your fears! Put yourself out there! Let the heat of challenging views and rigorous debate clarify and sharpen your ideas. At best you will have strengthened not only your position, but also your character, and if you fail, it would be, in the words of Theodore Roosevelt, “while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.”

I want my work to be excellent and able to withstand scrutiny, and if people aren’t willing to give it freely, I must wring it out of them. It’s my responsibility as a leader. I shout and bang my fist on my desk because I demand excellence from myself and my teams. I don’t know if we’ll ever get there, but through our pursuit, some awesome things are starting to happen. If testing is questioning a product in order to evaluate it, it is my opinion that questioning must start its focus on the questioner.

Recently, I have witnessed or been involved in several discussions and Twitter threads that seem to be equating challenging an idea with attacking the person who proposed the idea. I reject that premise as no idea, person or thing defines me – so it is impossible to offend me. But even if they did offend me, or I felt it was a personal attack – so what! Force it through the crucible of your own scrutiny and all the imperfections will burn away. Professional testers honing their skills should never shy away from challenges to their ideas – they should not just welcome, but court them!

I leave you these words to inspire you to seek your “crucible”:

“It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.”

Theodore Roosevelt, Excerpt from the speech “Citizenship In A Republic” delivered at the Sorbonne, in Paris, France on 23 April, 1910