Testing Testers

Software testing and testers

Necessity is the mother of all innovations, and here is one need that I am going to write about which is to test testers

  • How can we measure tester’s capability?
  • Who are the best testers?
  • How can we rate testers during interviews?

Above questions looks simple as hiring and promoting testers has been going on in the industry for years, today we have full fledge testing practice in place and specialized recruitment team to hire software testers. So have we matured enough? Do we have scientific methods to evaluate or test testers? Answer is “No” based on the research I did, I felt the need to spend time on this topic due to an experience I had with one of my client who hired 100+ testers from us but had given us really tough time because this client of ours was asking puzzles to our tester candidates during interviews and some of these puzzle were hard to crack.

There were many things that we tried to ensure our candidates are prepared for puzzles, we also had an unsuccessful attempt by having our learning partner look for trainer who can teach puzzle solving skills

Fundamentally do we want testers to solve puzzles? To me if need is to write algorithm like programmers then certainly problem solving skill is needed. Agreed testers need to know domain and basic technical skills to allow them to use the product on their own for product verification. Test Engineers are different breed altogether and my view of Test Engineer has been posted. I still stand by it.

In order to evaluate testers efficiently, we need to apply practical methods as opposed to theoretical interviews where softer skills and communication gets more inclination that actual tester’s testing capability.

Taking interviews is also a ‘testing’ activity, going by this notion, is interview the only technique to test testers? certainly not as we do come across candidate who clears interviews but fails to perform expected role. We had similar situation with the same client where candidate was interviewed multiple times by multiple people but later raised concerns on his capability and we had to arrange for replacement.

Here are few approaches to test tester that I could think of, would like to solicit more ideas

  • Practical Approach : Using a lab where candidates can test application/products and prepare a report, quality of report will determine candidate capability to test and articulate test results
  • Innovative thinking: Give specifications or show a product and check with tester how differently can users use it, a good tester would be innovative in coming up with different ways to use the product
  • Test result: Based on the interview, which is a form of test, candidate should be asked to provide honest feel if interview was “pass” or “fail”. Who else can be a better judge than a good Tester?