Follow OPM's guide to reference checking, which is excellent. See specifically the questions for references on page 5.
Some specific notes for our positions:
We should expect to get three good references. The Hiring Manager has some discretion here -- if a candidate only can supply two references, that may be acceptable, but only in special circumstances.
If a reference check turns up red flags, that's an immediate no-hire. Remember: these are people the candidate gave to us; we should expect to be hearing good things.
If a reference check can only confirm dates of employment, that's not a red flag (there may be legal issues which would prevent a good reference). But the reference "doesn't count"; return to the candidate for new references.