Ewance

How to Assess AI Fluency in Candidates and Teams

Anyone can say they are good with AI. Here is how to test it: what to assess, a work sample that allows AI, a review-the-output exercise, interview questions and a simple rubric.

Aharon Acosta, PhD

Reviewed by Aharon Acosta, PhD Founder of Ewance · Updated November 10, 2026

The basics

What to assess

AI fluency is the practiced ability to get good work done with AI: deciding what to hand to it, describing the task, judging the result and owning the outcome. That matches the four competencies of the AI Fluency framework by Dakan and Feller: delegation, description, discernment and diligence.

Because the skill shows in how someone works, the best way to assess it is to watch them work on a realistic task.

Test this

  • How they decide what to give to AI
  • How clearly they describe the task
  • How they check and improve the output
  • Whether they take responsibility for it

Do not rely on this

  • Tool trivia or model names
  • Prompt tricks
  • Self-rated skill levels
  • A test that bans AI altogether
Assess the judgment around the tool, not knowledge of the tool.

Why it is hard, and why it matters

Most people now use AI at work, but few employers can tell who uses it well. Claims on resumes are easy to write, and a polished AI-assisted application says little. At the same time, organizations that use AI have an obligation in the EU: Article 4 of the AI Act asks providers and deployers of AI systems to take measures so that their staff have a sufficient level of AI literacy.

Methods

Method 1: a work sample that allows AI

Give a short, realistic task, such as analyzing a small dataset, drafting a customer reply policy or summarizing a document for a manager. Allow any tools, and ask for two things: the result and a short note on how they worked, including where they used AI and what they checked or changed. Score the result and the note.

Method 2: review the AI's output

Give the candidate a piece of AI-written work that contains a few realistic mistakes, for example a wrong figure, an unsupported claim and a missing caveat, and ask them to review it for a colleague. People with good judgment find the errors, explain why they matter and fix them. This tests discernment directly and takes 20 to 30 minutes.

Method 3: structured interview questions

Ask every candidate the same questions and score them on the same scale. Questions that surface real practice:

  • Tell me about the last time AI gave you a wrong or misleading answer. How did you notice?
  • Describe a task where you decided not to use AI. Why?
  • Walk me through how you would use AI for [task from the role], step by step.
  • How do you check AI output before you rely on it?
  • How do you decide what information not to put into an AI tool?
  • Tell me about something you improved on after AI's first draft.

A simple rubric

Score each of the four competencies from observed behavior:

CompetencyStrong signalsWeak signals
DelegationSplits the task, chooses what AI suits and what not, explains whyUses AI for everything, or for nothing, without a reason
DescriptionGives context, constraints and examples; iterates when the result is offOne-line requests; accepts whatever comes back
DiscernmentFinds errors, checks sources, notices what is missingTrusts fluent text; cannot say how they checked
DiligenceSays where AI was used, protects sensitive data, owns the resultHides use, pastes private data, blames the tool

Add a note on how consistent the signals are across the task, the review exercise and the interview.

For teams

Assessing a team

To find out how fluent your team is, use the same ideas without the pressure of a hiring decision: give a representative task, ask people to document how they used AI and what they checked, and review the results together. Look for shared gaps, such as skipped verification or unclear data rules, and fix them with practice on real tasks and a short policy. Avoid self-rating surveys: people overrate themselves, and the results say little about behavior.

Watch-outs

What to avoid

Common traps:

  • Banning AI in the test: it removes the thing you want to see.
  • Testing tool trivia: tools change monthly, judgment lasts.
  • Using one clever prompt as a pass or fail.
  • Making the exercise long or unpaid.
  • Ignoring fairness: use the same task, time and rubric for everyone, and check local rules on selection and data.

Wrap-up

An example

Illustrative case: a marketing team hiring a content analyst sends each finalist a 25-minute exercise: an AI-written campaign summary with an invented statistic, an unsupported claim and a missing caveat. One candidate finds all three and explains how to verify each. Another rewrites the style but misses the invented statistic. In the interview, the first can describe exactly how they check AI output, and the second cannot.

Common mistakes

  • Assessing knowledge about AI instead of judgment with AI.
  • Relying on self-ratings.
  • Banning AI in the assessment.
  • Skipping a rubric, so scores reflect style.
  • Testing once and never revisiting, although tools and habits change.

Reference

Frequently asked questions

How do you assess AI fluency?
Give a realistic task with AI allowed, ask for the result and a note on how they worked, add an exercise where they review AI output with planted errors, and score the four competencies against a rubric.
What is AI fluency?
The practiced ability to get good work done with AI: deciding what to delegate, describing the task, judging the result and taking responsibility for it.
What are good interview questions for AI skills?
Ask about real experiences: a time AI gave a wrong answer and how they noticed, a task where they chose not to use AI, how they check output and how they protect sensitive data.
Should candidates be allowed to use AI in assessments?
Usually yes. A test that bans AI measures a situation they will not face at work. Allow it and assess the framing, checking and ownership around it.
How do you test AI skills in a team?
Use a representative task, ask people to document their AI use and checks, review the results together and look for shared gaps. Avoid self-rating surveys.
Does the EU require AI literacy at work?
Article 4 of the EU AI Act asks providers and deployers of AI systems to take measures to ensure a sufficient level of AI literacy among their staff. Check how it applies to your organization.

Sources

Find out who is AI-fluent on real tasks.