In April 2025, Shopify CEO Tobi Lütke published an internal memo that said, “Reflexive AI usage is now a baseline expectation at Shopify.” The memo also told teams to explain why AI could not meet a need before asking for more headcount or resources, and said AI usage would be added to performance and peer reviews.¹

That was an operating policy for Shopify. It was not a ready-made hiring standard for every company, or even for every job inside a company that uses AI heavily.

For hiring teams, the practical question is simpler: when AI fluency matters for a role, has the team defined what it means and collected enough evidence to assess it?

A line in a job description will not do that. The team has to connect AI fluency to real work, decide what level the role needs, and ask questions that show how the candidate uses AI, checks the result, handles failure, and knows when not to use it.

Metaview’s data shows that AI is already part of many interview conversations. Among 331,872 sessions where its topic taxonomy identified covered topics, 49,711 included probing classified as AI fluency, or 14.98%. A separate trend analysis found that direct discussion classified as “AI Usage” rose from 0.33% in Q3 2025 to 4.54% in Q2 2026. Those measures use different denominators and cannot be combined into one trend line.²

Neither measure says which companies had an AI policy, which roles required the skill, or whether candidates who showed it later performed well. Metaview’s corpus does not contain company-level mandate data or post-hire outcomes.

What "AI is baseline" actually requires from hiring teams

Declaring AI a baseline is easy. Turning it into a hiring criterion that two interviewers can apply in the same way is harder.

Start with the role, not the policy. Ask what the person should be able to do differently because AI is available. A recruiter might use it to research prospects, improve outreach, or turn an intake conversation into a clearer search brief. A data analyst might use it to explore a dataset, generate code, test an explanation, or automate a repeatable part of reporting. The tools may overlap, but the work and the risks do not.

Before adding AI fluency to the interview plan, agree on four things:

• the tasks or workflows where AI is genuinely relevant;
• how independently the person is expected to use it;
• the checks, safeguards, and approval points the work requires;
• the evidence that would show the workflow improved quality, speed, scope, or another role-relevant outcome.

This also prevents the interview from becoming a tool-name quiz. Products change quickly, and candidates have had unequal access to them. Someone from a regulated company may have worked under strict limits. Another candidate may have had a large budget and few restrictions. Neither situation, on its own, shows better judgment.

Assess what the candidate did, why they chose that approach, how they verified the output, and where they kept a human decision in the process.

Shopify chose a company-wide expectation for its own operating model. Other teams should set the bar from the work they need done, not from the fact that a well-known CEO published a memo.

Qonto’s approach makes sense for Qonto because AI is already part of how the company works. The same question would be less useful in a role where AI has little bearing on the actual job.⁵

The gap between policy and evaluation

Metaview’s corpus cannot tell you how many companies require AI fluency or whether every role that required it was assessed properly. The data has no company-by-company AI-mandate field, no normalized job-description requirement field, and no clean link between a stated requirement and every question asked in the interview.

It can show how often the topic appears in the conversations covered by its taxonomy.

Among 331,872 sessions with topic coverage, 49,711 included probing classified as AI fluency. That is 14.98%, or about one in seven covered sessions. The taxonomy is a proxy, so it may miss AI-related questions that were phrased or classified differently.²

A separate trend analysis tracked discussion classified as “AI Usage” from 0.33% in Q3 2025 to 4.54% in Q2 2026, an increase of 4.21 percentage points. This analysis uses a different denominator and interview set from the fluency-probing measure.²

The trend measure supports a narrow conclusion: direct discussion of AI usage became more common over that period. It does not show how many employers had an AI mandate, how many roles required the skill, or whether interviewers assessed it well.

It also says nothing about later job performance. Metaview’s corpus contains no performance reviews, retention, tenure, attrition, quality-of-hire, or other post-hire outcomes. Any claim connecting these interview topics to success after hiring would need a different dataset and a separate analysis.

What AI fluency looks like in an interview

A confident “I use AI every day” is not enough to rate. Ask for one recent, role-relevant piece of work and follow it from the original task to the finished result.

A useful opening question is:

“Walk me through a recent piece of work where AI meaningfully changed how you approached the task.”

Then probe the workflow:

• What were you trying to achieve?
• What information or context did you give the tool?
• What did you deliberately keep outside it?
• Why did you choose that model, tool, or approach?
• What did it get wrong, miss, or make harder?
• How did you verify the output before using it?
• What changed in the quality, speed, or scope of the work?
• What would you do differently next time?
• Where would you avoid AI or require a human decision?

The strongest evidence is usually specific and imperfect. A candidate can explain the first approach, the failure they noticed, the check that caught it, and the change they made. A polished answer that lists tools but never reaches a real workflow has not yet shown the competency.

Treat claimed gains carefully. “It reduced the work from four hours to one” is useful evidence about the candidate’s own account. It is not an independently verified result. For roles with a high AI-fluency bar, use a relevant work sample or another stage to test whether the workflow holds up in practice.

The two percentages come from separate analyses with different denominators. They should not be read as one series.

Updating your scorecard for AI competency

A scorecard field that says “AI proficiency: yes or no” leaves every interviewer to invent a standard. Set the required level before interviews begin, and define the evidence that earns it.

Metaview’s current AI-fluency framework includes three positive levels that can be adapted to a role. They are starting points, not universal grades:⁴

Add “not assessed,” “insufficient evidence,” and “required level not demonstrated” as separate options. Do not lower a candidate’s rating because the interviewer failed to ask the question or moved on before collecting enough detail.

The required level should depend on the job. A recruiter, account executive, data analyst, and engineering leader may use AI in completely different ways. Some roles may not need a formal AI-fluency rating at all.

A rubric can make the standard clearer and give interviewers a shared reference point. It does not prove that the competency predicts later performance. Testing that relationship would require properly governed post-hire data that Metaview’s corpus does not contain.

How Metaview surfaces AI fluency in interview conversations

Metaview’s Notetaker captures the conversation according to the team’s recording and consent settings, then produces a transcript and structured notes. A custom template can organize the factual evidence relevant to AI fluency, such as the workflow the candidate described, the checks they ran, the limits they named, and the outcome they claimed.³

Metaview does not decide whether those examples are strong enough. Its Notes do not evaluate, rank, compare, or recommend candidates. The interviewer reviews the draft against the transcript, adds context, and applies the scorecard rubric.

With supported ATS integrations, Metaview can autofill factual scorecard sections from what was said. It deliberately skips fields that require subjective evaluation, including ratings, hiring recommendations, culture fit, and overall impressions. The interviewer reviews and completes those fields before submission.³

Reports can help teams find the relevant conversations. Filters can narrow interviews by date, job title, role, stage, department, application status, topic, keyword, and other available properties. An AI column can extract the same factual item from each conversation, such as whether the candidate described a repeatable workflow and the example they gave. Teams can chart the results or export them to CSV.³

That extraction is a review aid. It is not a competency score. A yes-or-no column can show that a topic or example appeared. It cannot decide whether the example met the role’s standard.

A practical workflow is:

1. Decide whether AI fluency is relevant to the role and set the required level.
2. Give one interviewer clear ownership of the competency.
3. Use a notes template or factual scorecard field to capture the candidate’s workflow, checks, limits, and claimed result.
4. Review the draft against the transcript and apply the rating yourself.
5. Use Reports to check whether the topic was covered where expected and to collect examples for interviewer calibration.

Shopify’s memo is useful because it made the operating expectation explicit. For a hiring team, the next step is not to copy its wording. It is to decide what AI fluency means for each role, write that standard down, and collect evidence that lets interviewers apply it consistently.

Sources

¹ Shopify. Tobi Lütke’s April 2025 memo, publicly shared on 7 April 2025; and Shopify’s official follow-up, “Serious results, unserious methods: Shopify’s AI playground,” published 28 October 2025.

² Metaview canonical 2026 corpus analysis. QUESTIONS-3: 331,872 sessions with topic coverage, including 49,711 classified as probing AI fluency, or 14.98%. MARKET-3: the separate AI Usage topic rose from 0.33% in Q3 2025 to 4.54% in Q2 2026, a change of 4.21 percentage points. Both are proxy measures and use different denominators.

³ Metaview Help Center. “Best practices for Notes,” “Custom Templates,” “Reports Overview,” and current ATS integration documentation.

⁴ Metaview. “An AI fluency rubric for interviews, adapted from Zapier’s framework.”

⁵ Verified Metaview quote registry: Samy Aumar, Recruiting Operations at Qonto; and Elizabeth Calder, Vice President of People & Talent at The Bowdoin Group.