Sunday, March 19, 2006
As I've risen up the ranks, more and more often I am called upon to participate in interviews. I hate this, because I am always ill-prepared. There is usually no warning—the boss pokes his head in my office and asks me if I have fifteen spare minutes, and if I do, he hands me a resumé, introduces me to the candidate, and then I'm supposed to figure out if the candidate is a good one.
I'll note that it is not just my current boss who does this to me. Of all the interviews I've participated in during the last fifteen years, there was only one that I knew about before it happened.
I've made two hiring decisions myself during my career. One of those decisions was a good one, the other was not. I'd like to get a better-than-50% good-hire rate.
I've always wanted to sit down and prepare a few good interview questions, so that I'll be more prepared in these situations. My boss has a couple of really good programming exercises he gives to each candidate. Here's what is good about them:
- They involve writing actual code, so the candidates have to demonstrate actual skill, and can't just BS their way through the interview.
- If the candidate writes correct code immediately, then the boss is absolutely sure that the candidate is someone he wants. This saves a lot of time, as we can go straight to the "What salary do you want?" and "When can you start?" part of the interview.
- If the candidate gets stuck, the boss will offer hints and encourage the candidate to ask questions and talk through their thinking. This provides good insight into a candidate's reasoning skills and ability to discuss technical matters.
- About 90% of candidates can't get the right answers, even with assistance. This saves a lot of time, as the interview usually stops at that point. The exercises aren't very difficult, so anyone who can't do them is unsuitable for a programming position in our team.
I like these exercises, but they do have a couple of weaknesses. First, they are specific to C++, so they can weed out people who would be good assets, but haven't had a lot of experience with that particular language. Second, they deal with low-level implementation issues, so they are good at determining whether the candidate understands how computers really work, but not very good at discovering how the candidate approaches higher-level architectural and design issues.
So, I now have a goal of creating some incisive exercises for evaluating a candidate's higher-level abilities. I want exercises where the candidate has to do real coding, and where there are ample opportunities to talk about how they think. Here are my initial ideas:
- Give the candidate a bug report, and make the candidate fix the problem. I think this would be a good test of the ability to read code and diagnose problems.
- Give the candidate a new-feature request, and make the candidate implement it. I think this would be a good test of the ability to figure out how an application works, and to see how much of a mess they are going to make when they touch our code.
- Give the candidate some really bad code, and ask them to refactor it.
- Give the candidate a small set of unit tests, and have them write code that makes them all pass.
A problem with these ideas is that each one may take a lot of time. Another concern is that the candidates may not be able to use their preferred editors and programming environments, but I think any good programmer should be able to get by if offered Visual C++, Emacs, and Vi.
I don't want these exercises to be difficult. I want good candidates to breeze through them. They should be just difficult enough to catch the people who don't belong in our industry.