I have run hundreds of engineering interviews at Metaview. I have also been on the receiving end of dozens more. The thing nobody warned me about, the thing I had to learn the hard way, is that interviewing engineers is its own craft. It is not a side effect of being a good engineer, a good manager, or a good talker. It is a skill, and it has to be trained like one.
When I first started leading technical interviews, I treated them like conversations. I asked clever questions. I followed my gut. I left every loop with a strong opinion about the candidate and almost no evidence. Strong opinions, weakly held, became strong opinions, sloppily held. The scorecards I wrote afterwards were essays about how I felt, not records of what the candidate did.
What changed me was watching myself on tape, comparing my scorecards to better interviewers, and accepting that the candidate is not the only person being evaluated in the room. So is the interviewer. This post is what I would tell a new engineering interviewer on day one, including a younger me, about how to actually learn the craft. It is also a quiet argument for why structured capture matters more than personality, instinct, or pedigree.
Why I was bad at interviewing engineers
I was a competent engineer who liked talking to people. That combination convinced me I would be a good interviewer. It is exactly the combination that produces bad interviewers. Liking the candidate is not the same as evaluating them. Being able to talk about the problem is not the same as listening for whether they can solve it.
My early interviews drifted. I would ask a question, get an answer, and follow whichever thread interested me most. Two candidates with similar skill profiles would walk out of my room having taken completely different journeys. When I sat down to write scorecards, I could not actually compare them because I had not asked them the same things. I was running a conversation, not an evaluation.
The deeper problem was that I trusted my gut and called it signal. If a candidate reminded me of someone strong I had worked with, they were a "yes". If they fumbled the first question, I started looking for reasons to confirm the no. I am not unusual in this. Every interviewer I have ever trained at Metaview arrives with some version of the same bias, and the ones who get good are the ones who learn to see themselves doing it and stop.
Watching yourself on tape
The single most useful thing I did as a new interviewer was watch myself on tape. We use our own product for this, which means every interview I run gets a full recording, a transcript, and a structured scorecard. After my first ten loops, I went back and watched two of them end to end. It was excruciating, and it changed how I interview forever.
On the tape I noticed things I would have sworn I never did. I interrupted candidates mid-thought. I leaked the answer through follow-up questions. I asked three things in one sentence and then graded the candidate on which one they chose to answer. None of these were intentional, and all of them were patterns. The only way I knew was the tape.
The other useful move is the "virtual shadow". Instead of sitting through a live interview a senior person is running, watch their recordings on your own time, at 1.5x speed, skipping to the parts that matter. You see ten interviews in the time a live shadow gives you two, and you see the best of the best because the senior interviewer chose which to share. If you are training new interviewers and you do not give them recorded interviews to study, you are training them slower than you have to.
The interviewer is being evaluated in the room too. Most of them just do not realize it until they watch the tape.”
The first three minutes are the product
If I could go back and give my younger self one piece of advice, it would be this. The candidate's ceiling for the entire interview is set in the first three minutes. If you make them feel like they are being judged, they will play defense for an hour. If you make them feel like they are doing real work with a teammate, they will show you what they are actually capable of.
A good intro does four specific things. It puts the candidate at ease with a friendly, conversational opener. It outlines the structure of the interview so they know how long the task is, when they can ask questions, and what good looks like. It sells the company, because every candidate should walk out wanting the job whether or not we want them. And it starts the actual task on time, because eating into the work block makes a fair comparison impossible.
This sounds obvious written down. It is not obvious in the moment. I learned this by watching my own tapes back and noticing that my "warm intros" were five-minute monologues that ate into the candidate's solve time. The best interviewers I know rehearse the first three minutes the way a salesperson rehearses a pitch. The candidate's experience and your signal both depend on it.
Scorecards are the real training ground
The scorecard is where you find out if you actually saw the interview. I learned more from comparing my scorecards to senior interviewers' than from any other single training input. We would all write our scorecards independently, then sit down together and read them side by side. The places we disagreed were the places I had been making things up.
Sometimes the disagreement was about what the candidate did. I had remembered them solving the problem cleanly. The senior interviewer had noticed they got there only after a hint I had forgotten giving. Sometimes the disagreement was about what the bar should be. I would call something "strong" that two other interviewers called "fine". Sometimes I had picked up signal nobody else had, and I was right and they were wrong. All three were valuable.
The thing that made scorecard comparison work was that the underlying record was the same for everyone. We were all looking at the same transcript, the same recording, the same task output. A good interviewer and a bad interviewer can look identical on paper if all you have is a sticky note and a vibe. Put a transcript in front of them and the gap is obvious in five minutes.
What I do now vs what I used to do
The shift from gut-feel interviewing to structured interviewing is not a personality change. It is a workflow change. The framing below is what I would put in front of any engineering manager who is about to train a new interviewer at their company. The left column is what I used to do. The right column is what I do now, and what we train every new interviewer at Metaview to do.
- Improvise the intro, drift into chat, eat the task time
- Follow whichever thread interests me, ask different things to different candidates
- Take a few sticky-note jottings, write the scorecard from memory hours later
- Compare opinions in debrief, lose the loudest argument, call it consensus
- Rehearsed three-minute intro: ease, structure, sell, start on time
- Same task, same probes, same follow-ups across every candidate at the level
- Full recording and transcript captured automatically, scorecard written against evidence
- Compare scorecards before debrief, use disagreements as calibration not theater
If you are training someone new and you do not give them the right column to work with, you are asking them to relearn what every senior interviewer at every serious engineering team has already figured out. That is a waste of their time and yours.
Where AI gives engineering interviewers use
The reason I bring up AI here, instead of writing a purely retro post, is that the gap between gut-feel and structured interviewing used to require a heroic amount of manual work to close. Recording, transcribing, scoring against evidence, comparing across interviewers, training new people on real examples. You needed a dedicated talent operations function and a tolerant engineering team. That is no longer true. The work that used to require a team now runs in the background of every interview a recruiter or hiring manager already does.
Find engineering candidates who match what your strongest hires actually look like, not just the skills your job description lists.
Triage engineering applications against an Ideal Candidate Profile so the loop starts with the right people in the room.
Capture every engineering interview verbatim, surface the signal, and give the interviewer a scorecard draft against the rubric.
Track which interviewers calibrate well, which questions actually predict on-the-job performance, and where the loop leaks signal.
The point of these surfaces is not to replace the interviewer. It is to give the interviewer the same advantages a senior interviewer used to have only after years of reps. A transcript instead of fading memory. A scorecard draft against the rubric instead of a blank box. A pattern view across the whole team instead of one person's gut. LLM-assisted workflows are most useful where the human is already doing the work and just lacks the use to do it consistently.
The data backs this up. According to Metaview's 2026 AI & Hiring Alignment Report, surveying 505 recruiting leaders and hiring managers across North America and EMEA, the strongest hiring outcomes correlate tightly with strong recruiter-hiring-manager partnerships, and AI is the use that turns weak partnerships into strong ones.
Read those numbers in order and the story writes itself. 90% of teams say the relationship is good or excellent, but only the ones that actually live there beat their hiring goals. The gap between 79% and 36% is the gap between a recruiter and hiring manager who share a structured record of every interview and a pair who share a Slack thread and a memory. AI is the cheapest way to close it. The full report has the rest.
The operating shift
If you are an engineering leader, or you run hiring for one, here is what I would actually change on Monday morning. Not "audit your interview rubric" in the abstract. Three concrete moves you can sequence in the next two weeks. Each one is small. Together they compound.
One: record every engineering interview, with consent, and watch the first three minutes of three of them. Not a sample, not the highlights. Three full intros. You will see whether your team is setting candidates up to perform or to defend. You can fix the pattern in a single team meeting once you have seen it.
Two: standardize the interview rubric and have every interviewer write the scorecard against the same evidence. If your scorecards are essays, they are useless. If they cite specific moments from the transcript, they become a calibration tool, a training tool, and a defensible record. The same logic applies upstream: structured evidence beats personality everywhere in the loop.
Three: pair every new interviewer with a senior one for the first five loops, on tape, not in person. Have the new interviewer run the loop, have the senior interviewer leave comments on the transcript afterwards. You will train interviewers four times faster than the old "sit in the corner and take notes" model, and you will train them on the actual mistakes they made, not the ones the senior person remembered to flag.
Bring Metaview into your hiring stack.
Live notes, structured scorecards, and ATS sync - set up in under 10 minutes.
Frequently asked questions
How long does it take to actually get good at interviewing engineers?
In my experience, a deliberate ramp of five to ten loops with tape review and scorecard comparison beats two years of unsupervised interviewing. The variable is not reps, it is feedback density. Without recordings, transcripts, and someone senior comparing notes with you, the reps mostly compound your bad habits.
Should engineers run technical interviews, or should hiring managers?
Both, on the same loop, against the same rubric. The engineer reads the technical signal, the hiring manager reads the team-fit and trajectory signal. The mistake is letting either of them write a scorecard from memory. Capture the interview, write against evidence, calibrate together.
Is it worth recording engineering interviews?
Yes, with candidate consent. The recording is the only training input that actually shows the interviewer what they did. Memory is too unreliable to train against, and live shadowing scales linearly in the worst possible way. Recordings let one strong interviewer train ten new ones.
How do you keep the candidate experience strong while running a structured loop?
Structure and warmth are not opposites. A rehearsed three-minute intro is warmer than an improvised five-minute one, because the candidate knows where they stand. Tell them what is happening, why it is happening, and what will happen next. The interviewers who are best at signal are usually the ones candidates rate highest on experience.
Where does AI actually help in engineering interviews?
In the connective tissue. Transcription so the interviewer is not splitting attention. A scorecard draft against the rubric so the writeup is anchored to evidence. Pattern views across the team so you can see which interviewers calibrate well and which questions actually predict performance. AI is not the interviewer. It is the use that makes a good interviewer consistent and a new interviewer trainable.
