Recently, I've been asked by my Director to help interview a number of Program Manager candidates. In the last two days, I've interviewed four different people. There was another flurry of candidates a few months ago in which I also participated, but we didn't end up hiring any of them.
Now I have a tremendous amount of experience in hiring developers. I've been responsible for staffing large teams in short order on at least three different occasions now. My rough guess is if you combined the people I've phone screened or interviewed in person, the number is somewhere around the three hundred mark. I've probably greenlit upwards of forty of fifty developers. This is just in the past three years mind you. Needless to say I have formed some very definite thoughts about the interview process for developers in this time.
What's interesting, though, is that as concrete and refined as my thoughts are when it comes to what constitutes a good developer interview (I'll leave such thoughts to a subsequent blog entry), I don't really have any similiar set of defined criteria for other positions. It ends up being much more 'intuition' based, which frankly bothers me.
When interviewing a developer, I have expectations that the candidate understands basic data structures, that when presented with one or more problems he can work through them logically and methodically, even if he can't quite arrive at the right answer. If, for example, he doesn't know what a linked list is or a hash table, and the situations in which one of the two data structures would be used over the other, he's going to have a really difficult time getting a thumbs up from me. If he can't solve some simple (and I mean SIMPLE) exercises in his preferred language, then he really is not getting through.
These past few days, though, have really highlighted that I don't have any similar questions for other positions. What I've done for these past few interviews with program managers is to look over the resume, look for things with which I'm familiar and spot check the candidates
knowledge there. Yesterday, for example, I had a Program Manager with an extremely impressive resume. It listed his workat numerous, huge companies, with all sorts of awards.
I also noticed that his education was in programming and that first six years of his career were spent as a Java developer in Java. So, I asked him to explain inheritance in a few sentences. I don't even recall exactly what his answer was as it was mostly gibberish. He certainly he didn't mention the idea of using the criteria of 'is a' to determine whether it a particular inheritance scheme was appropriate, or that inheritance, along with encapsulation and polymorphism, is one of the fundamental concepts of object-oriented programming. Nor did he say anything about using inheritance to help reduce code duplication by centralizing shared functionality in a base class from which other classes could be derived. Any of these would have been acceptable. I wasn't looking for a text book answer, just something that gave me an indication he knew what inheritance was at some basic level.
He couldn't do it.
He also listed Scrum on his resume, and so I gently asked him to give a basic explanation of that as well. Where one might expect him to mention the various roles within a Scrum team, such as Product Owner, Scrum Master, things like Sprint planning sessions, the daily Scrum, or retrospectives, he gave me a vague answer about how it 'helped focus on past and future requirements' and some other drivel.
In fact, I was happy that he had these things on his resume. He choose to list them, thereby making them fair game, and could provide a basic explanation of neither. I felt quite justified in justified in excluding him from future consideration.
More often, however, there is nothing so concrete on a PGM's resume that I can spot check. If the person doesn't have a development background, if he doesn't have any experience with Scrum (or another program management methodology with which I'm familiar) I end up having to rely almost solely on hypotheticals and questions about his past roles, which I still don't like much at all. The problem is such questions are (relatively) easy to answer.
Not to say that many candidates don't still do a miserable job with them, but if you're asking what a PGM would do when the client is making unreasonable demands, it's all well and good for them to say "I'd stick to my guns and refer them to the SOW, but see if we could accomodate their request reasonably with minimal risk"
When they have someone at the client company yelling at them saying how they missed some requirement or that the company promised this feature, though, it's much less likely they'll act that way though. Like many things, it is easy in theory, but in practice tend to be horribly difficult.
I may ask the person what was the most challenging project he worked on and why? What are the characteristics of a well-run project (beyond hitting milestones), and what the characteristics of a poorly run one? I may pose questions based on current difficulties faced by the team and ask him what he would do?
But the problem is, none of this really gives me any great sense the person is qualified. At best, it disqualifies him if he can't give a plausible, well-reasoned answer, or if he veers totally off topic. But a "good" answer guarantees almost nothing. I'm still struggling with the question of whether there are "good" ways to evaluate a potential PGM beyond a reference from someone who's previously worked with the person (which is the best thing) or hypotheticals like this, which really aren't all that useful.
The things is I think a *good* PGM can contribute immense value to a project. A poor one thought simply create work and stress, derails process, and is a huge liability to a team. And unfortunately most PGMs in my experience are poor ones. Bureaucrats more concerned with action items and holding status meetings that with thinking critically about the projects to which they are assigned and how to intelligently manage scope and the customer.
That what makes the good ones even more remarkable and valuable. I just wish I had a better way to find the good ones.
Error'd: Tangled Up In Blue
2 days ago
4 comments:
I feel you man. The very little times that I've interviewed Pgms I tried to focus on how they specifically would deal with teams of developers (as that's my only experience).
What happens if the estimates are off? How do they enforce estimates/targets? Do they pad estimates? Do they gauge the general accuracy somehow?
How do they prioritize the requirements that actually get implemented?
Even then... I suppose all of these probably have good "textbook" answers. So my preference is to hear them speak from actual experience, rather than "Well... what I would do is..."
Peopleware has a great chapter called "Hiring a juggler", which makes a great point:
http://www.kdedevelopers.org/node/444
"It would be ludicrous to think of hiring a juggler without first seeing him perform. That's just common sense. Yet when you set out to hire an engineer or a designer or a programmer or a group manager, the rules of common sense are often suspended. You don't ask to see a design or a program or anything. In fact, the interview is just talk"
In effective dev interviews, we ask interviewees to whiteboard or do actual coding, to weed out the "good talkers but bad doers." Why couldn't we apply this same general principle to hiring project managers?
Maybe, and this is a bit of a lateral idea...for interviewing project managers, a portion of the interview should involve real-world exercises: Have the interviewer type a response to a tricky email thread (test their written communciation and diplomacy). Have them provide estimates for a sample software project. Test them with a sample conversation with an angry client.
Basically, the idea is to make the interview as close to the real work as possible. I have no illusions this will ever be adopted in real-life interviews...hopefully, just good food for thought.
I think that's the right approach. The difficulty is that in such instances you're again stuck with hypotheticals. A design is a design is a design.
But an emial thread in which the client is known to be fictitious and, won't be angrily calling your boss if you stand up, isn't quite the same as the real thing. Maybe I'm being too pessimistic, but againt so much of the response is situational that I think it's hard to replicate it with anything but the real world.
I guess in my "perfect world", every PGM would have a trial period to see how they really responsd. Of course, the difficulty with doing that would then be that once you have someone, it's easy to settle. you've brought them onboard, they've ramped up, "good enough" ends up being the enemy of great...and you're stuck with yet another crappy PGM.
But then I'm jaded.
I'm a huge fan of the scenario based interview. You can learn not just from their answers, but in how they approach the exercise. Do they try to game the system, do they improve the scenario with questions, do they get into the role?
Some of these observations are merely for secondary characteristics but attention to detail, the ability to improvise, and the ability to control a situation should be observable in a scenario based interview. These aren't the only aspects a candidate should be judged on but they are high on my list.
Post a Comment