Detailed engineering selection process guide

A guide to the engineering interviewing process, from start to finish. This is a guide for anyone who needs to run the process, or for process nerds who want to know how it works from A-Z.

Most bits have detailed HOWTOs; see the introduction for links to those other guides.

Roles and responsibilities

These are the roles and responsibilities of the folks involved in selection. Ideally, these would be stable through the entire hiring cohort.

  • Talent Lead (TL) - from the Talent team
    • leads the hiring from the Talent team
    • process manager, overseeing the process from start to finish
    • primary point of contact for candidates
    • liaison with HR
  • Hiring Manager (HM) - the Engineering Director, or delegate
    • leads the project from the Engineering team
    • together with the Talent Lead, selects SMEs for resume reviews/interviews
    • responsible for position-specific selection guides (if they don't already exist)
    • makes a final hire/no-hire call based on feedback
      • (or submits a recommendation to the Director, in the case of a delegate)
    • with the Talent lead, conducts a post-cohort retrospective
    • may or may not also be an SME
  • Subject Matter Experts (SMEs)
    • typically from the engineering chapter, but really anyone from 18F with expertise and availability to help with selection
    • review resumes
    • conduct interviews
      • and submit written recommendations to the Hiring Lead
    • SMEs may cannot review a candidate's resume and also interview them -- we need different SMEs for each step.
  • Senior Interviewer (SI)
    • one of the SMEs
    • a more experienced interviewer - should have conducted both Tech and Core Values interviews in the past, preferably a few of each
    • conducts final interview with every candidate (combined tech/core values)
    • ideally, the same person for all candidates. if time doesn't permit, this could be 2 people
    • could be the same person as the Hiring Manager
  • Phone Screener (PS)
    • will be one of the engineering supervisors or the director
    • conducts initial phone screen

A word on cohorts

We collect review applications in cohorts: this means open applications for a limited time (typically about a month), then close the application and review all applications together. Once we select candidates from the cohort, if we have more seats we can run another cohort.

Any changes to the process or guides should happen between cohorts (probably: as the result of a retrospective on the cohort). Of course, if there are egregious errors, mid-stream corrections are OK, but as much as possible we should strive to present the same selection process to every candidate.

This helps ensure consistency and fairness into the process. It also lets us collect statistics and compare results between cohorts, letting us take a more data-driven approach to hiring.

Selection Process

1. Before starting interviews

Before we start any selection we should:

  • Select interviewers (SMEs)
    • We need 3 SMEs for resume review, and 4 for interviews. These can't be the same people for each candidate. So this points to the need for a fairly large pool. 8-10 is probably a minimum for most positions.
  • Select Senior Interviewer(s)
  • Develop interview guides for the specific position (if they don't already exist)
  • Hold a training session with SMEs to go over the interview flow, interview guides, and such
    • May want to conduct a mock interview for interviewers to observe
    • New interviewers: should observe at least 1-2 interviews before conducting one themselves
  • Start selection!

2. Resume screen

  • TL will assign 3 reviewers per resume
  • They can be the same as someone who’ll interview later
  • They can’t have referred the candidate
  • Reviewers will score according to resume scoring guides, and will leave feedback and scores in a form.
  • Based on scores, TL+HM will construct a list of Yes, Maybe and Nos:
    • Stack-rank based on reviewer votes
    • Choose appropriate cut-offs, depending on numbers (definitely phone screen all consensus yeses, and potentially some mixed yes/maybes, depending on numbers.)
    • Communicate rejection to everyone on the “No” list
    • Schedule phone screens with “Yes”es
    • Hold “Maybes” for after all the "Yes"es get interviewed, potentially interview in a second round (if needed)

3. Phone Screen

  • TL will reach out to candidate, schedule phone screen with PS
  • Phone screen is a short (20-30) minute pre-interview, covering:
    • how the process will work from here
    • looking for fundamental core values fit
  • Those who pass: on to interviews
  • Those who don’t pass: TL will communicate rejection immediately

4. Pre-work

  • TL reachers out to assign pre-work
  • Each candidate gets some technical pre-work to bring to the first technical interview. For most positions this is a simple coding assignment.
    • Some positions may allow candidates to bring their own code samples instead (if, e.g., they have open source they can share)
  • Pre-work is due to the one day before the first interview (to give the interviewer time to prepare)

5. Interviews

  • TL schedules interviews
  • Each candidate will get 4 interviews:
    • Tech interview #1
      • review pre-work, ask follow-up questions
      • time permitting, ask questions from tech interview guide
    • Tech interview #2
      • question from the position-specific tech interview guide
      • may overlap with previous questions (this is OK)
    • Core values interview
      • questions from the shared core values interview guide
    • Final interview
      • conducted by Senior Interviewer
      • Covers both tech and core values, and wrap-up with candidate
  • Most of the time, all interviews should be conducted. However, if there's a disaster, interviewers can reach out to the TL+HM and suggest "pulling the plug". See the interview guide for details.

Notes on interview format:

  • interviews are 1:1, no paired interviewers
    • panel interviews are intimidating, and can magnify mistakes (candidates don't get another "shot" to recover)
    • exception: "training" interviewers may sit in to observe, but if so they should be muted and video off, just an observer
  • interviewers should ask questions off relevant guides, no one-offs
    • overlap with previous interviews is just fine (even, good - gives a candidate a chance to recover )

6. Decision time

  • Interviewers submit feedback in writing to TL+HM (see guide below for format)
  • If the results are especially controversial, hold a meeting to discuss
  • HM makes final hire/no-hire decision
    • Consensus isn't required, nor is there any voting. The HM makes the final call.
  • "No Hire" decisions: communicate immediately to the candidate
  • "Hire" decisions:
    • HM checks references
    • hold for the final cohort review
  • Final cohort review:
    • TL+HM (+Director, if applicable) review all "hire" decisions
    • If there are too many: time to make some tough decisions about who to offer to first.
  • Selected candidates go to HR for offers!

7. Cohort retrospective

  • Conduct a retrospective on the hiring cohort, incorporate feedback into next round