An artificial intelligence (AI) tool has rated every application for the role. Most of the ratings look right, but a candidate you’d have moved forward sits near the bottom, and one near the top looks wrong for the job as it stands today. The queue is still full, so the easy move is to follow the rating and keep going.

This guide is for recruiters using AI to screen candidates, whether your tool ranks applications or sorts them into groups by how well they fit. It shows you when to go with the AI’s rating, when to override it, and how to record each override so it improves the next round of screening.

A rating measures how closely the application matches the role’s profile: the criteria for the role that you and the hiring manager agreed on and gave the tool. It can’t see anything outside the application and the profile. So my advice is to treat ratings as first reads: check a rating whenever the profile may be out of date or the application may leave something out, and write down a reason every time you go against it.

Using AI to screen candidates starts with what the rating can see.

Before you decide whether to trust a rating, know what it’s built from. The tool compares each application with the role’s profile, which is usually drawn from the job description. That tells you where a rating can mislead you:

  • It’s only as current as the profile. If the role has changed since intake, the tool is still matching candidates to the old version.
  • It’s only as complete as the application. A skill the candidate has but didn’t write down can’t count toward the rating.
  • It looks for experience in the form the profile describes. A candidate who built the same skills by a different route can rate lower than they should.
  • It can’t ask a follow-up question. Anything you’d only learn by talking to the candidate is missing.

Whatever the rating says, whether the candidate moves forward is still your decision.

In this panel discussion hosted by Hung Lee of Recruiting Brainfood, talent acquisition leaders from Cleo and EMnify talk through where human assessment works best in hiring, and where AI is best placed to help:

If you’re still setting up the screening itself, from knockout questions to writing the profile, start with this practical guide to AI resume screening.

Decide when to go with the rating and when to override it.

When your own read and the rating agree, go with the rating. The cases below are where your read and the rating tend to split. For each, the table shows what the rating is reflecting and what to do about it.

The case What the rating reflects What to do
Rated high, but the role has changed since intake A match against criteria that no longer describe the job. Read the application against the role as it is now, and override if it no longer fits. Then agree on an update to the profile with the hiring manager, so new applications are rated against the role as it stands today.
Rated low, with an unusual career path Experience that doesn’t follow the route the profile describes. Read the whole application. Override if it shows the skills the role needs, and write down which ones.
Rated high, but something about it worries you What the application shows, with nothing yet from a conversation. Don’t reject on a hunch. Move them to a screening call, ask about what worries you, then decide.
A must-have the application doesn’t mention either way No evidence, which isn’t the same as a no. Don’t count it as a fail. Ask about it on the screening call.
You keep overriding the same kind of candidate A profile that doesn’t describe what the hiring manager wants. Stop overriding ratings one candidate at a time, and change the profile.

The missing must-have can point back to the profile too: if applications keep leaving the same must-have unmentioned, the profile may be asking for something candidates rarely put in an application. There’s a separate guide on writing must-haves an application can prove.

Every case in the table comes down to a reason you could say out loud, and the next step is writing that reason down.

Write down a reason for every override.

An override is only useful if someone can read why it happened. Without a reason, no one can tell a good call from a personal preference, and that includes you a few months later. Record the reason at the moment you decide:

  1. Write one sentence on what the rating missed: a skill in the work history, a change to the role, or a must-have with no evidence either way.
  2. Tie the reason to a criterion in the profile, or name the criterion the profile is missing.
  3. Record it next to the decision in your applicant tracking system (ATS), where the hiring manager and the rest of the team can read it.
  4. If you can’t write the reason in a sentence, don’t override yet. Ask the hiring manager, and decide once you can say why.

A written reason also helps keep screening fair. An override with no reason leaves room for bias, because no one can check it. One with a reason can be checked, by you or by the hiring manager.

Change the profile when the same override keeps coming back.

An override fixes one decision. When you find yourself making the same one again and again, the problem sits in the profile, and overriding ratings one candidate at a time won’t fix it. Use the reasons you’ve written down to find what needs to change:

  1. At the end of each week of screening, read that week’s override reasons for the role.
  2. Group the reasons that say the same thing: a skill the profile underrates, or a requirement the role no longer needs.
  3. Take each group to the hiring manager and decide together whether the profile should change. The hiring manager owns what the role needs, so the change is theirs to approve.
  4. Change the profile, then read a few of the next ratings before you trust them again.

A profile that matches the role cuts the number of overrides you need, and the ones left are the cases that call for your judgment.

Where Metaview fits.

The steps above work best with a screening tool that does two things: it shows the reasoning behind each rating, so you can see what the rating rests on, and it leaves every decision with a person.

Metaview is the Agentic Recruiting Platform, and its Application Review reads every application against the role’s Ideal Candidate Profile. It drafts that profile from the job description and the context you give it, and you review, edit, and approve the profile before any application is rated. Each application then goes into a fit group, from Great down to Poor, with the reasoning behind it.

An Application Review list of applicants in Great, Good, and Okay fit groups, with one application opened to its reasoning beside the recruiter’s own Reject and Progress candidate buttons, and a custom column for startup experience
Fit groups in Application Review, with the reasoning for one application and the recruiter’s own “Reject” and “Progress candidate” buttons. The candidates shown are sample data.

When you and the hiring manager approve a change to the profile, Application Review rates every application again against the new version. Your written reasons then show whether the change fixed what it was meant to.

Metaview never rejects a candidate on its own. The recruiter makes every call to progress or reject. Each progress or reject decision goes back to your ATS if it’s Ashby, Greenhouse, Lever, or SmartRecruiters.

Siadhal Magos, Metaview’s co-founder and chief executive, put the role of AI in screening this way:

Application Review represents a fundamental shift in how AI should work in recruiting, not as a replacement for human judgment, but as infrastructure that handles operational scale while keeping recruiters focused on what matters: connection and decision-making.”
Siadhal Magos Siadhal Magos Co-founder and Chief Executive Officer · Metaview

One more part of Application Review is worth knowing before you act on a rating. You can add a column that checks every application against a criterion the profile doesn’t cover, like the column for startup experience in the screenshot above.

When a case in the table sends a candidate to a screening call, Metaview’s Notetaker can join that call as a visible participant. Where consent has been given, it records and transcribes the conversation, then turns it into structured notes. The recording captures every spoken word, so the candidate’s answer to what worried you, or to the must-have their application didn’t mention, is there in their own words when you decide.

See the reasoning behind every rating.
Book a demo to see how Metaview Application Review rates every application against your Ideal Candidate Profile, shows why, and leaves every decision with your recruiters.
See it live

Check your overrides after each role.

When a role closes, look back at its overrides. They show whether the ratings and your team’s judgment are working together, and what to fix before the next role opens.

  1. Count the overrides in each direction: candidates moved forward against a low rating, and candidates rejected despite a high one. If almost all of them go one way, find the criterion in the profile that keeps rating those candidates too low, or too high.
  2. Count the overrides with no written reason. If there are any, write down a reason for every override on the next role, from the first one.
  3. Check how the candidates you moved forward against a low rating did at the screening call. If most were rejected there, the reasons behind those overrides didn’t hold up, so read them again. If most went through, the profile underrates something, and your written reasons show what it is.

After a few roles, you’ll know which ratings to go with, every override will carry a reason, and the profile will describe the role your hiring manager is hiring for.

See it in action.

Review every AI rating with its reasoning in view.

See how Metaview Application Review shows why it rated each application the way it did, and leaves the decision to progress or reject with your team.

Frequently asked.

Should a recruiter ever reject a candidate on the AI’s rating alone?

Not without reading the application. A low rating says the application doesn’t match the profile well. Read it, and if it doesn’t show what the role needs, reject it with that reason.

Who should be allowed to override the AI’s rating?

Whoever owns the screening decision for the role, usually the recruiter. The hiring manager approves any change to the profile. Whoever overrides a rating, recruiter or hiring manager, writes down the reason.

How often should recruiters override the AI?

There’s no right number. Overriding rarely is fine if the ratings match what the hiring manager wants. Overriding often for the same reason means the profile needs changing. Overriding often for different reasons, with none written down, means the decisions need a closer look.

Does overriding the AI bring bias back into screening?

It can, when the override rests on a feeling no one can check. A written reason tied to what the role needs lets you or the hiring manager review each override, so an override that rests on personal preference is easy to spot.

What should I do when the AI keeps rating good candidates low?

Find what those candidates have in common, then look for it in the profile: a requirement that’s too narrow, experience described one way when the role accepts several, or a criterion the role no longer needs. Agree on the change with the hiring manager and update the profile, instead of overriding the same kind of candidate every time.