I spend a lot of my week working alongside AI agents, and the thing that surprises people is how much the results vary by who's driving. Hand the same agent to two engineers and one gets a finished feature back while the other gets a confident mess. The gap between someone who can direct an agent well and someone who can't has started to look less like a skill gap and more like a seniority gap.

Companies have noticed. Shopify told its teams that reflexive AI use is now a baseline expectation, and plenty of others have quietly added some version of it to their hiring bar. The trouble is that “works well with AI” has become a line on the scorecard that almost nobody knows how to actually interview for, so it gets waved through on a confident answer and a tool name.

Working with an AI agent isn't really a tool skill. It's a management skill: scoping the work, handing it off, checking what comes back, and knowing when the agent is confidently wrong. You already know how to interview for that when the report is a person. This is how to do it when the report is an agent, with a four-level rubric and the questions that surface each level.

Why working with agents is a hiring bar

Two people with the same title and the same resume now produce very different output, and a lot of the difference comes down to how well they work with AI. One delegates a chunk of the work to an agent, checks it, and ships more. The other does it all by hand, or worse, ships whatever the agent handed back without reading it closely. Over a quarter, that compounds into a real gap in what each of them gets done.

That's why teams have stopped treating this as a nice-to-have and started writing it into the bar. The instinct is right, and it leads somewhere uncomfortable: if you're going to screen for it, you have to be able to tell the levels apart, and most interviews can't. It helps to be clear-eyed about what an agent does, and doesn't do, to a person's output.

That's the whole case for interviewing carefully. An agent multiplies whatever judgment the person already has. Give a strong operator an agent and they get more done with sharper calls about what to trust. Give it to someone without that judgment and you've automated their blind spots. So you're not really hiring for AI use. You're hiring for the judgment the agent will amplify.

The tools your team already touches are turning into agents too. Our own product is built as a set of agents that work across hiring, and the same rule holds there: they reward the people who know how to direct them.

Test for the workflow, not the tool

The usual interview for this is one question, “do you use AI?”, and a nod at the answer. That sorts almost nobody. A candidate can name three agents and still not be able to get good work out of one, and the most confident answer in the room is often the emptiest. Here is the difference between the version that sorts people and the version that doesn't.

The trap is rewarding the person who is most enthusiastic about AI. Enthusiasm is easy to perform, and judgment isn't. The teams that get this wrong end up hiring the candidate who was busiest with AI rather than the one who got the most out of it, which is its own version of an old measurement problem:

Lots of AI activity isn't the same as real progress with it. The best operators draw the line between using AI to go faster and using it to do better work, which is the distinction worth listening for in an interview:

The four-level delegation rubric

A rubric beats a vibe. Four levels do most of the work, and they describe how a person works with an agent rather than which apps they've opened. Read across the row to place a candidate: how they work with agents, what you'll hear when they're at that level, and what kind of role it clears.

The jump that matters sits between Delegator and Supervisor, and it isn't about tools. It's oversight: whether the person treats the agent as confident and often wrong, and builds that assumption into how they work. A delegator trusts the output. A supervisor checks it on purpose, and can tell you exactly where it tends to go wrong.

The rubric only works if your questions surface real behavior instead of rehearsed opinions. Three moves do most of that:

1. Ask for one handoff. “Walk me through the last real piece of work you handed to an agent, end to end. What did you hand off, and what did you keep?” A lived handoff exposes the level; a hypothetical just rewards whoever talks about AI best.
2. Inspect the guardrails. How did they scope it, what context did they give, and what did they check before trusting the result? A delegator stops at “it gave me a draft.” A supervisor can narrate how they kept control.
3. Probe the catch. “When did an agent do something confidently wrong, and how did you notice?” The answer separates someone who trusts the output from someone who supervises it.

Write down what a strong, average, and weak answer looks like for each level before the interview, and assign the questions to one interviewer rather than all of them. Spread across a panel, this turns into four people asking “so, do you use AI?” and nobody going deep.

Score it from the room, not from memory

A rubric is only as good as the evidence behind each score. The usual failure is that the interviewer asks a sharp question, hears a great answer, and then writes the scorecard two days later from a fuzzy memory of it. The level rounds up, and the rubric becomes decoration.

This is where capture earns its place. Because Notetaker captures every spoken word, the candidate's real example, the work they scoped and the failure they caught, is on the record instead of in your head. The scorecard then drafts from the conversation against the rubric you set, so the delegation level is filled in from what the candidate described, ready for you to confirm or adjust.

From there, Reports shows whether interviewers assessed the competency where they were supposed to, and how the level breaks down across the pipeline and by team. A rubric that lives only in one interviewer's head drifts. One you can see across every loop holds.

It also keeps interviewers honest with each other. You can see where a strong yes from one person means something different from another's, and calibrate before that gap quietly costs you a hire.

None of this replaces your judgment. It gives the judgment something solid to stand on. And the reason it's worth the effort is that the teams treating AI as core to how they work are the ones pulling ahead on the numbers that matter:

Those figures come from Metaview’s 2026 AI & Hiring Alignment Report, surveying 505 recruiting leaders and hiring managers across North America and EMEA. Hiring people who can get real work out of an agent is the other half of that same advantage.

What this means for your team

Pick one role you're hiring for right now and decide, out loud with the hiring manager, which level it genuinely needs. A junior or highly specialized role might only need a delegator. A role you're betting the function on needs a supervisor or an orchestrator. Writing that line down is most of the work, and it's the part teams keep skipping.

Then put the rubric where the interview happens. Add the questions to that loop, build the bar into your question bank and scorecard templates, and let the capture layer record the answers so the score is evidence, not recall. Connect it through native integrations and keep the ATS you already run. If you want the groundwork first, our writeups on great interviewers, quality of hire, and interview quality set it up, Reports keeps the bar honest across the team, and pricing shows what it costs.

Working with AI agents stopped being optional faster than most hiring processes adapted. You don't need a perfect science to keep up. You need a shared definition of the levels, a few questions that surface them, and a way to score that doesn't depend on who was in the room. That's how you interview for the skill that's quietly separating your strongest people from everyone else.