References are either the highest-signal moment in your entire hiring process or the most expensive checkbox exercise in modern recruiting. The difference is not the candidate. The difference is whether the hiring manager runs the call like a serious interview or treats it as a formality. Siadhal Magos and Nolan Church spent an episode taking the reference call apart and rebuilding it.

Siadhal Magos (CEO and co-founder of Metaview) joined Nolan Church on 10x Recruiting (more episodes on the 10x Recruiting hub) for a rare back-and-forth on how to actually run a reference call. The framing question: the Zapier CEO publicly does 20 references per executive hire. Is that founder theater, or the right bar? The answer is closer to "neither," and the conversation walks through the system Nolan and Siadhal actually use.

This recap covers the framework: how to set expectations with the candidate, how to flip the reference's psychology toward honesty, the two questions that surface real signal, why three to seven good references beats twenty mediocre ones, and the back-channel ethics most hiring teams get wrong.

References are the highest-signal data point in hiring

Both Nolan and Siadhal kept coming back to the same observation. Of every signal in the hiring process (the interview loop, the work sample, the take-home, the values check), the reference call is the only one where the person grading the candidate has actually watched them do the job at scale, over time, in real conditions. Nothing else gets close to that fidelity.

The catch is that most reference calls extract zero of that fidelity. The hiring manager asks generic questions, the reference gives generic praise, and the call is over in fifteen minutes with nothing learned. The reference becomes a tax, not an input.

The fix is structural, not effortful. Better questions, better framing of the call's purpose, and a posture that signals you're not looking for a yes; you're looking for the truth.

Front door vs back door: set expectations upfront

The work starts with the candidate, not the reference. Nolan's exact framing from minute one of any senior search:

The candidate gets to name the references they want you to call (front door) and gets to name the people you should not call and explain why (back door). The "why" line on the people-to-avoid list often reveals more than every front-door call combined. A candidate who says "please don't call my last manager because we had real friction over comp" is giving you the truth before the references even start.

The act of being explicit about both lanes also changes how the candidate behaves through the rest of the process. They know you'll check. They calibrate.

The party question that actually works

The single best question Nolan uses on a reference call is deliberately informal. It is designed to disarm.

The frame matters. "If I saw you at a party" pulls the reference out of HR-call mode and into casual mode. People answer behavioral-economics questions more honestly when they don't feel they're on the record. The reference, instead of defending their friend, starts running the actual failure scenarios. "Honestly, if it didn't work, it would probably be because…", and the real risk surface gets articulated.

Siadhal's complement: before any reference call, position yourself as the person trying to set the candidate up to succeed, not the person grading them. The frame is partnership, not evaluation. The references open up.

The percentile-ranking move

The second high-yield question is sharper. Nolan asks the reference to rank the candidate against everyone else they have worked with in a comparable role. Where references say "top 5%" or "top 1%" without hesitation, you have a real signal. Where they hedge, you have a different signal.

Then he asks the follow-up: what is the gap between this candidate and the best person you have worked with at this level? The answer reveals the candidate's ceiling, not their average. Most references will pause, think, and then say something specific and useful. That specificity is the actual data point.

The third tell is whether the reference table-pounds for the candidate. Lines like "if you don't hire him, I will in six months" are the gold standard. They reveal genuine enthusiasm rather than polite recommendation.

Three to seven beats twenty

The 20-references-per-exec hire move that triggered the episode is, in Nolan's framing, "founder porn." The math doesn't work. You cannot get 20 deep, brutally honest, hour-long reference conversations done in the window between offer and start. What you get instead is 20 fifteen-minute calls with diminishing signal per call.

The better target is three to seven references where the conversation actually goes deep. The criteria: people who have direct work experience with the candidate, who have managed them or been managed by them, and who are willing to talk for longer than the surface-level call.

For senior hires, the depth matters more than the count. A 90-minute call with someone who managed the candidate for three years is worth more than a panel of 20 quick takes from people who briefly worked alongside them.

Back-channel ethics: when references can burn careers

Back-channel references are the highest-risk lane in the process. Most senior candidates have not told their current employer they are looking. A careless back-channel call can destroy a career, full stop. Always get permission. Always ask who to avoid. Always operate on the assumption that the candidate has trusted you with confidential information.

Nolan's creative-but-clean back-channel pattern: reach out to an investor in the candidate's current company and ask broadly who the best operators are in a given function. Triangulate without naming the candidate. The signal comes back without the candidate's situation being exposed.

The other rule: hiring managers run their own back-channel calls. An external recruiter cannot interpret what they hear because they do not know the role, the culture, or the operating context. Delegating this step is how good candidates get killed by careless intelligence.

Where AI gives recruiting teams use

Reference calls are where most recruiting teams under-invest because the work is laborious and the output is hard to systematize. AI changes that equation by surfacing the questions worth asking and capturing the signal worth keeping. The goal is not to replace the reference call. The goal is to make every reference call count.

Metaview Notetaker captures both the interview loop and the reference call so the conversation becomes searchable structured data, not a faded memory. Reports surfaces patterns across interview signals, including the gaps where a reference question would close the loop. For example, if no interviewer covered "how does this person handle disagreement with their manager," the system flags it as a reference question worth asking. Application Review handles the volume work upstream so the senior team has time to actually do references properly. For the AI-as-pattern-recognition layer on top of unstructured interview data, see claude-for-recruiters, and for the interviewer-quality angle that sets up better reference questions, see good-interviewer-bad-interviewer.

Numbers from Metaview's 2026 AI & Hiring Alignment Report, surveying 505 recruiting leaders and hiring managers across North America and EMEA. The 3.8x lift matters for reference calls because the reference is where recruiter-HM alignment shows up most visibly. If the HM does not know what was tested in the interview loop, they cannot ask the reference question that closes the loop. AI that captures the loop properly converts what was tribal knowledge into shared infrastructure between recruiter and HM.

The operating shift

Three concrete moves to upgrade your reference process this quarter:

One: rebuild your reference call script around the party question and the percentile rank. Throw out the generic "tell me about a strength and a weakness" template. The party question and the percentile ranking are the two highest-yield prompts in the entire reference toolkit. Make them mandatory on every senior hire.

Two: hire-manager-owned, not recruiter-delegated. The hiring manager runs the calls, takes the notes, and translates the answers into onboarding plans. The recruiter coordinates and supports. Inverting that ownership is how the signal gets lost.

Three: capture the interview signal upstream so the reference questions write themselves. Most reference calls fail because the hiring manager does not know what was tested in the loop. Fix the upstream data capture and the reference questions become obvious.

Teams that get this right come out of the loop with three or four references' worth of operating context, not just a thumbs up. That context is what makes the first 90 days go right.