How to run a calibrated shortlist review with hiring managers

A shortlist review is a short meeting where the recruiter and the hiring manager look at the best candidates side by side and agree who moves forward. Make it work by agreeing the criteria first, sharing the evidence in the same format for each candidate, letting each reviewer score before the meeting, and recording the reasons. This guide shows the format, the agenda and a template.

Why shortlist reviews go wrong

The usual problems are easy to spot. The recruiter sends a long list of CVs with a few lines of text. The manager skims and picks on first impressions. Feedback arrives late, or in a vague message: “not quite right”. The recruiter has to guess what the manager wants, and the next round repeats the problem. A good review turns this into a clear, short, fair discussion.

Step 1: agree the criteria first

Before you source or screen, hold the intake meeting and write the must-haves and nice-to-haves. Add a scoring scale. A simple one works: zero for no evidence, one for weak, two for clear, three for strong. Make sure everyone who will review knows the scale. If requirements are loose, the review turns into an argument about taste. Our guide on checkable job requirements helps.

Step 2: choose a format for the shortlist

Use the same one-page format for each candidate. Keep it short so reviewers read it.

Section	What to include
Header	Candidate ID or name, source, stage, key dates
Requirement evidence	For each must-have: the evidence found, its source (CV line, screening call note), and a score
Questions to ask	Gaps or claims to check at interview
Practical points	Notice period, location, pay expectation, right to work status where known
Recruiter note	One or two lines on what stood out, factual only

Do not include photos, age, family or other personal details that do not relate to the job. If you use a tool to help prepare the evidence, check each line yourself before sharing. FastHire is coming soon and is designed to show each requirement next to the exact CV passage that supports it. People make the decision.

Step 3: score separately before the meeting

Send the shortlist 24 hours ahead. Ask each reviewer to score each candidate on each requirement, and to write one line for any score of zero or three. Ask them to send scores to you before the meeting, not in a group chat. This avoids anchoring, where the first opinion sets the tone.

Keep the group small. Two to four reviewers is enough: the hiring manager, the recruiter, and perhaps a team member who will work with the person.

Step 4: run the calibration meeting

Calibration means checking that everyone uses the scale in the same way. A “two” for one reviewer should not be a “three” for another. Use the meeting to discuss differences, not to repeat the CVs.

1. Start with the criteria. Read the must-haves aloud (one minute).
2. Show the scores in a grid, with differences highlighted.
3. For each candidate, start with the largest score gap. Ask each reviewer what evidence they used.
4. Agree the score or note that the evidence is unclear and a question is needed.
5. Rank the shortlist in groups: move forward, hold, do not move forward.
6. Agree the interview questions and who will ask them.
7. Confirm next steps and dates.

Candidate	Reviewer A	Reviewer B	Gap	Discussion note
Candidate 1	2	3	1	B saw a result figure that A missed
Candidate 2	3	1	2	Different views on recent experience
Candidate 3	2	2	0	Agreed

Step 5: set decision rules

Agree how decisions are made before the meeting. For example: a candidate moves forward only if they score at least two on every must-have, or have a clear plan to learn a missing one. A candidate with a zero on a must-have goes to a “hold” list until the team agrees whether the requirement is truly needed. The manager makes the final call, and the recruiter records the reasons.

Be careful with the phrase “not a culture fit”. It hides bias. Ask what behaviour or requirement it refers to, and score against that instead.

Step 6: record decisions and reasons

After the meeting, send a short summary within one working day.

• Who moves to interview, who is on hold, who is not moving forward.
• The reason for each decision, in terms of the requirements.
• Open questions for each interviewed candidate.
• Interview dates and panel.
• Any changes to the criteria, with who agreed.

These records protect fairness and speed up feedback to candidates. See how to give feedback to rejected candidates and rejection emails that respect candidates.

Fairness and legal points

• Use job-related criteria only. See the EEOC list of prohibited practices and the GOV.UK employing people pages.
• Treat candidates with a disability fairly and offer adjustments. See reasonable adjustments in interviews.
• Store scores and notes securely, and delete them when the retention period ends.
• If you use automated tools to prepare the shortlist, check local rules and keep a person in charge.

Agenda template for a 30 minute review

Time	Item	Owner
0-3 min	Criteria and scale recap	Recruiter
3-10 min	Score grid and differences	Recruiter
10-22 min	Discuss the largest gaps, candidate by candidate	All
22-27 min	Group decisions and interview plan	Hiring manager
27-30 min	Next steps and dates	Recruiter

Handling difficult situations

The manager wants someone who does not meet the must-haves

Ask which requirement the manager wants to change, and why. If the requirement is not needed, update the brief for all candidates, and re-score. If it is needed, explain the gap. Do not make an exception for one person without changing the rule for everyone.

The manager has a favourite from the start

Use the scores and notes. Ask a second reviewer to read the same evidence. Remind the group that the aim is a fair comparison. If the favourite is the best fit, the scores will show it.

The shortlist is too short

If fewer than three candidates meet the must-haves, check the requirements, the pay range and the sourcing. See how to source candidates and push back on unrealistic requests.

The manager is slow to respond

Agree a response time in the intake meeting. Send a short reminder with the deadline, and offer a fifteen minute call to talk through the shortlist.

Worked example (fictional)

A recruiter sends a shortlist of five candidates for a finance analyst role. Each has a one-page summary with evidence for the four must-haves. Two reviewers score on their own. In the 30 minute meeting, they find that Candidate 2 has a gap of two points on “recent budget experience”. The manager says the candidate ran a budget three years ago, while the other reviewer thought it was current. They agree the evidence is dated and add an interview question. They move three candidates to interview, hold one, and decline one, and the recruiter sends a summary the same day.

What a strong evidence summary looks like

The one-page summary is the heart of the review. It should let a busy manager see in one minute how a candidate meets each requirement. Below is a fictional example for a finance analyst role. Notice that each line points to evidence and the source, and none relies on adjectives.

Requirement	Evidence found	Source	Score
Builds monthly forecasts	Built a rolling twelve month forecast for three business units	CV, current role	3
Uses advanced spreadsheet functions	Describes lookup and pivot use for monthly close	CV, screening call	2
Explains numbers to non-finance staff	Presented results to sales leads each month	Screening call	2
Experience with budgets in a team of 50+	Not shown	CV	0

The last line shows why the format works. It invites a question for the interview instead of a rejection. If the candidate has the experience, they can show it. If not, the manager sees the gap clearly.

Keeping the review short and useful

• Send the shortlist early, with the deadline for scores.
• Keep each summary to one page.
• Limit the meeting to 30 minutes, and use the time on disagreements.
• Use a shared document, not email threads, so the record is in one place.
• End with decisions, owners and dates.

A good rhythm is one review per batch of candidates, ideally within two days of the last screening call. Long gaps between screening and review cause candidate drop-out. Our guide on how to reduce time to hire shows where delay usually hides.

Training managers to be good reviewers

Not all managers know how to review a shortlist. Spend ten minutes teaching three habits.

1. Read the evidence against each requirement before forming an overall view.
2. Score alone first, and write one line for extreme scores.
3. Name the evidence when you disagree. “I do not feel it” is not evidence. “The role needs X, and the CV shows Y” is.

Share a short guide at the intake meeting, and repeat the points at the start of the first review. After two or three reviews, most managers pick up the routine.

Records, privacy and data protection

The shortlist contains personal data. Share it only with people who need it. Remove contact details when the manager does not need them yet. Delete or archive files when the process ends, following the rules for your country. See our guides on GDPR for job applicants and how long to keep rejected applicant data.

Checklist: before you send a shortlist

• The must-haves and the scoring scale are written and agreed.
• Each candidate has a one-page evidence summary in the same format.
• Personal details that are not job-related are removed.
• Questions to ask are noted for every gap or claim.
• The deadline for scores and the meeting time are set.
• The reviewers know how to score and to record their reasons.
• A shared place exists for scores, notes and decisions.

Checklist: after the meeting

• A decision summary has gone to all reviewers within one working day.
• Candidates who moved forward have a date for the interview.
• Candidates who did not move forward receive a kind message, and feedback if they reached interview.
• Scores and notes are saved with the candidate file.
• Any change in the criteria is recorded and applied to all candidates.
• Lessons for the next review are noted: what was unclear, what took too long, what worked.

If you repeat this routine for every role, reviews become quicker and fairer. Managers learn what is expected, candidates move through the process with less waiting, and you build a record that shows how decisions were made. That record is useful if you need to explain a decision, defend a process or train a new recruiter.

One last tip: keep a short log of your own review habits. After each meeting, note how long it ran, how many scores differed by two points or more, and whether any candidate was moved after discussion. Over a few roles, the log shows whether your criteria are clear and whether your managers are learning the routine, and it gives you facts to use when you ask for more time or support.

Frequently asked questions

How many candidates should be on a shortlist?

Three to six is common. The number depends on the role and the volume. Quality of evidence matters more than the number.

Should the hiring manager see all applications?

For small volumes, yes. For large volumes, the recruiter screens against the agreed criteria and shows the manager the shortlist plus a summary of how many were screened and why.

How do I keep managers from choosing on gut feeling?

Agree the criteria first, collect scores separately, and ask for evidence behind every strong view.

Do I need a meeting for every shortlist?

Not always. For simple roles, a written review with scores may be enough. Use a meeting for senior, hard-to-fill or high-volume roles.

Related guides

• Recruiters hub
• Intake meeting with the hiring manager
• Employer guides

This is general information, not legal advice.