This is a guide for interviewers conducting engineering interviews. Everyone who's conducting interviews in engineering should read this guide so that as much as possible we can present a fair, consistent interview process to all candidates.
To sum up:
- The golden rule: your goal is find a reason not to hire (but don’t be a jerk!)
- When you start, tell the candidate what to expect
- Please follow the guides…
- … but not dogmatically: feel free to dig deep.
- Take good notes: behavioral is best
- After the interview, submit a written "hire/no-hire" recommendation, with justification
- If there’s a disaster, considering ending interviews early
Click each section for more details.
The golden rule: your goal is find a reason not to hire (but don’t be a jerk!)
Ultimately, your goal in the interview is to find a reason not to hire. This sounds rough, but there are several good reasons to approach an interview this way:
Hiring the wrong person is a much worse outcome than not hiring a good fit. Research bears this out: toxic employees are a massive drain on an organization, producing much more harm than the best employees produce good. Also, it’s far easier to undo a “no hire” than it is to undo a failed hire.
There are always more qualified applicants - 18F has a high profile, and we don't struggle to find good candidates. So, we can afford to be super-selective.
If you approach an interview looking for a reason to hire someone: you’ll find it. Everyone has good qualities; they wouldn’t have made it to interview if not.
If, on the other hand, you approach an interview looking for a reason not to hire, and don’t find any — well, that’s a pretty great indicator that you’ve got a fantastic candidate.
Now, this isn’t an excuse to be a jerk! Being selective is important, but so is being nice. We can have high standards without being mean about it. This means:
Be friendly and conversational. This isn’t an interrogation; it’s a two-way conversation. It’s entirely possible — easy, even — to be casual and conversational while also asking tough questions. You do it every day when you interact with your colleagues and discuss work topics; pretend the candidate is one of those co-workers.
It’s OK to ask tough questions, dig deep, and be skeptical — but absolutely not OK to mock or belittle someone for a weak or incorrect answer. If someone gives an answer that’s weak or that you disagree with, just take a note and move on. Especially watch out for subtle forms of putting people down for lack of knowledge (i.e. no feigning surprise).
Forgive a first mistake. In an hour of interviewing, it’s almost a certainty that a candidate will make a mistake. Maybe they’ll get a technical thing wrong; maybe they’ll have a weak answer to one of your questions; maybe they’ll refer to engineers as “guys”. If this happens once: no big deal! Everyone makes mistakes. But, take a note, and watch for it to happen again. And compare notes with other interviewers at the end. Single mistakes are no big deal, but a pattern is a problem, so watch for it.
If you come to a conclusion very quickly, in the first few minutes of the interview, spend the rest of the interview trying to disprove that snap judgement. Snap judgement are much more likely to be prone to bias than considered ones. We tend to jump to conclusions and then look for evidence to support our hypothesis. To compensate, if you find you’ve reached a Yes/No conclusion very quickly, spend the rest of the interview trying to disprove that hypothesis. Explicitly look for evidence that you’re wrong. If you’ve decided immediately that a candidate is not qualified, ask targeted questions and spend rest of the interview trying as hard as you can to find evidence that they are qualified.
Kicking off the interview
Before you begin each interview, remind yourself to watch out for unconscious bias. Remember that we’re especially susceptible to assume that underrepresented minorities -- women, people of color, etc. -- are less qualified than their white male counterparts. Be wary of “gut decisions” that aren’t backed up by specific behavior you’ve noted during the interview (this is one reason we ask behavioral questions, and try to take notes behaviorally).
When you first get on the call with the candidate, it’s a good idea to spend a quick minute explaining how the interview will go, and what the interviewee can expect. Here’s the script I use:
Hi, I’m Jacob; I’m a Supervisor on the Engineering team at 18F.
Thanks for interviewing with me today. This’ll be a behavioral interview, which means I’ll ask a series of questions about experiences you’ve had and how you handled them. There are no “right” answers; I’m interested in talking through these situations with you. I’ll ask maybe 5-7 questions, and this will take us about an hour, perhaps a bit less. Don’t be surprised if others ask the same questions in other interviews; that’s normal.
There’ll be times when I ask for more information, or want to dig deeper into your answers. That’s normal, too: I want to understand what you did and why.I’ll be taking notes, please don’t let that distract you.
I’ll ask you my questions first, and then I’ll leave some time to answer any questions you’ve got for me. I’m excited you’re here - let’s get started!
Please follow the guides…
We have guides; please follow them!
- FIXME: guide links here
These guides exist for a reason: by asking all candidates the same questions, we can can try to get consistency. The questions in the guides are behavioral in nature, designed to get candidates talking about specific situations and how they handled them. Guidelines are suggested for evaluating the strength of the answers.
There are two more questions not in the guides that you might want to ask:
“Tell me about yourself”: this is a great opening question, and gives a candidate a chance to “sell” themselves. It can also suggest which follow-up questions from the interview guide will be the best to ask next, and can guide the direction of the interview from then on out. If you ask this question, it’s probably appropriate to spend maybe 5-10 minutes on it.
“Do you have any questions for me?”: this is a good final question; it gives candidates a chance to ask you something they might be wondering. It can also reveal some of a candidate’s values. I like to leave about 5-10 minutes at the end of the interview to ask this question.
Whenever possible, ask these questions exactly as they’re worded to try to get consistency between multiple candidates.
… but not dogmatically: feel free to dig deep.
It’s important to start with the guides… but the real meat of the interview is in how you follow-up, dig deep, and ask questions. Don’t hesitate to interrupt if a candidate is going on a tangent you don’t have time for: they’ll be thankful for you keeping them on track.
Follow-ups are also how you get technical information out of behavior questions. Push candidates to get super-specific about their tool choices, how well they know/use those tools, what their experiences were, etc.
Because these questions are behavioral, it’s important that you really understand a candidate’s answers. If they tell you about a decision they made, don’t just leave it at that: make sure you understand why they made that decision, how their decision-making process went, what other options they considered, what they’d do differently in the future, and so forth. You should probably be spending 5-15 minutes on each question, between letting the candidate answer and your follow-ups.
This means you definitely won’t get to all the questions in the guides. This is fine! I typically start with the same couple of questions, but then let the candidate’s answers determine where I go next.
Check the candidate documents before the interview begins: you can review the candidate’s response, and the phone screener may have left notes about what to dig for in interviews; this can help you choose your questions wisely. (These notes should also be in your calendar invite.)
Take good notes: behavioral is best
When interviewing, please take notes -- the more detailed, the better. Try to note what the candidate says, rather than your impressions — that will help you share behavioral reasons for your conclusions and decisions. That is, try to write down what the candidate said or did, rather than how you felt about it. For example:
Good: “candidate ‘forced engineers to do \$X’” (captures what a candidate said)
Bad: “candidate has an adversarial relationship with engineers” (interpretation of what they said above)
Good: “candidate has maintained an \$X as an open source module for 2.5 years” (captures what a candidate did)
Bad: “candidate is a good open source maintainer” (your feelings/interpretations of what they’ve done)
When the interview is done, please summarize your notes and submit a written recommendation:
Submit a written "hire/no-hire" recommendation, with justification
After you interview a candidate, you'll need to submit a written recommendation to the hiring manager. This should consist of two parts:
- A "hire" or "no-hire" recommendation. Yes, we're trying to force you to make a hard call. If you're not sure, that's a "no-hire".
- Justification for that recommendation, based on what you observed during the interview. Please be as specific as possible, and include as much detail from your notes as you can. Again, focus on what the candidate said or did, rather than how you interpreted it.
If the written feedback is unclear or especially controversial, the Hiring Manager may schedule some time to talk over the feedback.
If there’s a disaster, considering ending interviews early
If something really huge went wrong, or if there’s a massive red flag, talk the Hiring Manager and Talent about possibly ending interviews early.
This is a pretty rare and extreme thing; someone would have to exhibit truly out-of-line behavior to consider this. For example: repeated sexist behavior, telling an obvious and egregious lie and doubling-down when challenged, showing up to an interview intoxicated, etc (these are all things that have actually happened to me). Something this bad is rare, but if it happens it’s not worth continuing.
If you get a massive red flags: ping Talent and the Hiring Manager and we’ll decide if we should stop at this point.