Can I see you do that?

I wouldn't have an interview process that didn't include an audition. I've done about ten tester interviews with auditions now, and it tells you a lot about the candidate. You really see how they think.

For the audition, choose a small program to test. You can write it yourself or use something already out there. It doesn't have to be elaborate. You might add a pretend spec or readme.txt alongside the program.

The most interesting thing is how many different approaches people take testing the program. If ever you needed a rationale for having a team of testers, this is it. Get several people looking at software and you'll get some very different approaches. Some people dig into the spec and take it as the only truth. What it explicitly states is law, and what it doesn't mention is forbidden or unimportant. Others dive in and start typing in values for the sides of triangles. Some try special characters. Everyone does a lot of boundary testing. Some have a UI testing eye and make notes of proper tab order or improper resizing. None have run all the test Meyers notes in his book.

One of the best parts is at the end when I ask how much of the program they have tested on a scale of 0 to 100. Candidates have taken twenty to forty five minutes to test the program. I've gotten answers ranging from "10" to "90" (not correlated with longer or shorter times testing).

There is a lot of talk about interviewing and how imprecise it is. I don't subscribe to the "Blink interviewing hypothesis" entirely, having seen what an audition can show. While an audition won't tell you how hard the candidate will work, it does give you an idea of where they are at. Are they too trusting of authority, be it a spec or a developer? Do they run down all the leads they find while testing or stop at the first bug found?

At the PNSQC conference this year, Jon Bach gave a great presentation about traps testers fall into. In his presentation Jon lists the "Top Ten Tendencies that Trap Testers":

#10 Stakeholder Trust
#9 Compartmental Thinking
#8 Definition Faith
#7 Inattentional Blindness
#6 Dismissed Confusion
#5 Performance Paralysis
#4 Function Fanaticism
#3 Yourself, Untested
#2 Bad Oracles
#1 Premature Celebration

These are really good traps to consider when evaluating candidates or yourself.