Interview calibration is how a hiring team gets its interviewers to read the same evidence against the same bar. A split panel, where your interviewers disagree about a candidate, is where you find out whether they’re calibrated. What they disagree about tells you where the problem is: the candidate, the interview, or the way your interviewers rate.

When a panel splits, it’s tempting to settle it the fast way: take a vote, average the ratings, or let the most senior interviewer decide. That gets you a verdict on this candidate, but it hides why the split happened, and the same split comes back on the next one. That’s why my advice is to find the cause first, before anyone argues about the verdict.

This guide is for recruiters and hiring managers who run interview panels. It shows you how to find where your panel split and why, and how to decide on the candidate once you know the cause. Then it covers how to use a month of splits to calibrate your interviewers.

Get every verdict in before anyone compares notes.

A split only tells you something if each interviewer’s view is their own, so collect every view first. Before the debrief, ask every interviewer to:

  • submit their scorecard, with an overall recommendation and a rating for each requirement they assessed (the skills and qualities the role calls for, such as communication)
  • write the evidence behind each rating, in a sentence or two about what the candidate said or did
  • keep their view of the candidate to themselves until everyone on the panel has submitted

Once an interviewer has heard a colleague’s view, their own scorecard tends to drift toward it, and you lose an independent read. Collecting scores before any conversation prevents that, so any split you find later reflects what each interviewer saw in the interview.

For how to set up the panel itself, including how each interviewer scores before the debrief, read this guide to panel interview best practices.

Find where the panel split.

With every scorecard in, and before the debrief, find the exact place where the interviewers part ways. Two overall verdicts that disagree tell you there’s a split, but not what it’s about, so compare the scorecards requirement by requirement. Take these four steps:

  1. Lay the scorecards side by side, with a row for each requirement of the role and a column for each interviewer.
  2. Mark each row where interviewers who assessed the same requirement gave it different ratings.
  3. Mark each row no one rated, and each rating with no written evidence behind it, since a split can come from evidence the interview never collected.
  4. For every marked row, put the interviewers’ written evidence next to each other.

An overall split often comes down to one or two requirements. If two interviewers agree on everything except communication, communication is the only requirement the panel needs to resolve.

Trace each split to its cause.

Each split has a cause, and the cause sets the fix. Read the evidence on each marked row, then match the split to one of these five causes:

Cause How to spot it What it tells you
Different readings of the requirement The interviewers rated the same requirement, but their evidence is about different things The requirement needs a clearer definition for this role
Different readings of a rating level The interviewers cite the same answer but give it different ratings The descriptions of your rating levels, such as what a 3 means, are too loose to apply the same way
Missing evidence A rating has no evidence behind it, or no one asked about the requirement The interview didn’t collect what the decision needs
A different bar for the role The ratings mostly agree, but the overall verdicts don’t, or the interviewers expect different seniority The panel hasn’t agreed what the role needs
An uneven candidate Each rating is well supported, but the interviewers who say hire assessed the candidate’s strengths and those who say no assessed the gaps The candidate is strong in some areas and weak in others

If no row is marked and only the overall verdicts differ, start with the last two causes in the table: the panel may hold different bars for the role, or the candidate may be uneven.

The first four causes come from the interview or the interviewers, so fixing them helps with the next candidate too. Only the last comes from the candidate.

Fix the cause, then decide on the candidate.

Once you know why the panel split, match your next step to the cause, and decide on the candidate last:

  1. If the interviewers read the requirement differently, agree with them on what it means for this role, then ask each of them to rate it again from their own evidence.
  2. If they read a rating level differently, read the answer together against the description of each rating level and agree which one it meets.
  3. If evidence is missing, get it before you decide, with a short follow-up conversation or a question for the reference check.
  4. If your interviewers hold different bars for the role, agree with the hiring manager on what the role needs, then have the panel read the scorecards again against that.
  5. If the candidate is uneven, weigh the gaps against what the role needs most. Leave the call to the hiring manager, and write any gap you accept into the decision as a known risk.

Once the cause is dealt with, take the scorecards, with any ratings that changed, into the debrief, where the hiring manager makes the call. For how to run that meeting, read this guide to running an effective interview debrief.

Run interview calibration from the month’s splits.

One split shows you a problem on one candidate’s panel. A month of them shows you where your interviewers read the bar differently, which is the problem interview calibration fixes. Once a month, take these three steps:

  1. List every split from the month’s panels, with the requirement it was on and the cause you traced it to.
  2. Group the splits by cause, and look for a requirement or a rating level that keeps appearing.
  3. Fix the cause that comes up most often, using the list below.

Fixing a cause for one candidate settles that candidate. These fixes change the interview for every candidate after, one for each of the first four causes:

  • Where interviewers read a requirement differently, write down what it means for the role and share it before the next round of interviews.
  • Where interviewers read a rating level differently, rewrite its description and add an example of an answer at that level. For help with the wording, read this guide to writing interview rubrics.
  • Where evidence keeps going missing, make sure every requirement has an interviewer assigned to it, with a question that gets at it.
  • Where the panel keeps disagreeing about the bar, settle it with the hiring manager at the intake for the next role.

Sometimes the pattern is one interviewer rather than one cause. If one interviewer’s ratings keep sitting apart from the rest of the panel, run a calibration session with them: have them and another interviewer rate the same recorded interview, then compare the ratings and the evidence behind each. A separate guide covers how to run a calibration session and read ratings by interviewer.

You’ll know calibration is working when you see fewer interviewer-caused splits: more of the splits you trace come down to an uneven candidate, instead of the interview or the way your interviewers rate.

Where Metaview fits.

Tracing a split, on one candidate or across a month, is hard when the only record of an interview is the scorecard written afterward. Two interviewers can write up the same answer so differently that you can’t tell what the candidate said. It gets easier when every interview leaves a record of its own.

Metaview is the Agentic Recruiting Platform, and it keeps that record. Where consent has been given, the interview is captured by the Metaview Notetaker. It records and transcribes the interview, then turns the conversation into structured notes.

Interview notes in the Metaview Notetaker, grouped under topic headings, each heading marked with the number of sources behind it and followed by bullet points
Notes from an interview, grouped by topic. The candidate’s answers shown are sample data.

The recording captures every spoken word, so when two interviewers rate the same answer differently, you can read what the candidate said and see which rating the answer supports.

Read what the candidate said when interviewers disagree.
Book a demo to see how the Metaview Notetaker records each interview, so your team can check a split against the answer itself.
See it live

The Notetaker also drafts the scorecard from the conversation, so each rating’s evidence starts from what the candidate said. The interviewer then reviews the draft, rates the candidate, and submits the scorecard.

While you trace a split, the Metaview Assistant answers questions in plain language from your captured interviews, so you can ask it what the candidate said about the requirement the panel split on.

Whatever the record shows, each interviewer’s rating is one input to a decision the hiring manager owns. Shahriar Tajbakhsh, Metaview’s co-founder and chief technology officer, has put it this way:

Good interviewers know they are contributing to a hiring decision. Bad interviewers believe they are making a hiring decision.”
Shahriar Tajbakhsh Shahriar Tajbakhsh Co-founder and Chief Technology Officer · Metaview

An interviewer who treats their scorecard as a contribution has less reason to defend their rating when the panel disagrees, and more reason to show the evidence behind it. Tracing a split to its cause depends on that evidence.

See it in action.

Check every split against what the candidate said.

See how the Metaview Notetaker records each interview, so your team can trace a split to the answer behind it before anyone makes the call.

Frequently asked.

What is interview calibration?

It’s the work of getting your interviewers to assess the same requirements against the same bar, so two of them who hear the same answer give it the same rating. Some teams also use the term for role calibration, where the recruiter and the hiring manager agree on the profile before sourcing starts.

How is interview calibration different from inter-rater reliability?

Inter-rater reliability describes how consistently different raters score the same thing, and the term comes from research. Interview calibration is the practical work that makes your interviewers more consistent: clearer requirements, clearer rating levels, and a regular look at where they disagree.

Should the most senior interviewer break a tie?

No. Seniority doesn’t tell you which rating the evidence supports. Trace the split to its cause first, then let the hiring manager make the call from the evidence.

What if your panel always agrees?

Check that each interviewer submits before hearing anyone else’s view, and that each one assesses the requirements they were given. A panel that always agrees may be repeating one interviewer’s view instead of giving several.

Should interviewers see each other’s scorecards before they submit?

No. Each interviewer submits first, so their scorecard records their own read. Once everyone has submitted, comparing the scorecards is how you find the split.