When two interviewers score the same candidate very differently, the usual instinct is to trust the more senior one. That instinct is often wrong.
Scoring agreement is not a function of experience. It is a function of structure, and the teams with the most aligned panels are the ones that built the alignment deliberately.
Here is why interviewers disagree, why seniority does not fix it, and what actually raises agreement across a panel.
Why interviewers disagree.
Most disagreement is not really about the candidate. It is about the inputs. Two interviewers who asked different questions, against different private definitions of good, will reach different scores about a perfectly consistent candidate.
Add in impressions standing in for evidence, and the gap widens. The candidate becomes a screen onto which each interviewer projects their own bar.
The seniority myth.
It is tempting to assume the most experienced interviewer is the most reliable. Experience does bring pattern recognition. It also brings conviction, and strong convictions held privately are a recipe for disagreement, not alignment.
Two senior interviewers, each confident in a different mental model, can disagree just as sharply as two juniors, sometimes more, because each is sure they are right. Seniority without shared structure does not converge.
Structure is the real lever.
When everyone asks comparable questions and scores against the same rubric, with shared definitions of what a 3 versus a 4 means, agreement rises because the inputs are finally the same.
Calibration is the other half. Reviewing real examples together, and aligning on where the bar sits, is what keeps a panel converged over time as memories drift.
How to raise agreement?
Treat agreement as something you build, not something you hope your senior people provide.
- Standardize the core questions and the rubric for each role.
- Agree what each rating actually means before the loop runs.
- Calibrate regularly with real examples, not abstract principles.
- Capture interviews so a disagreement can be traced to evidence, not competing memories.
Do that and agreement stops depending on who is in the room. It starts depending on the process, which is the only version that scales.
Bring Metaview into your hiring stack.
Structured interview notes, draft scorecards, and ATS sync.
Frequently asked.
What makes interviewers agree on candidate scores?
Shared structure: the same questions, a clear rubric, agreed definitions of what each rating means, and regular calibration. Agreement is built by the process, not assumed from experience.
Do senior interviewers agree more than junior ones?
Not reliably. Seniority brings pattern recognition but also strong, idiosyncratic opinions. Without shared structure, two senior interviewers can disagree just as sharply as two junior ones.
Why do interviewers disagree on the same candidate?
Usually because they asked different questions, are scoring against different definitions of good, or are weighting impressions rather than evidence.
How do you improve interviewer scoring agreement?
Standardize questions and the rubric, calibrate the panel regularly, and capture interviews so disagreements can be traced to evidence rather than memory.
