Audit a Hiring-Screen Classifier for Fairness Across Cohorts
Overview
What this challenge is about.
Audit a classifier for fairness across cohorts using 8,000 CVs, run counterfactual tests, and produce an audit report. Get a verifiable certificate.
The scenario
The vendor (around 80 staff, around 200 enterprise customers in EMEA) is responding to the EU AI Act's high-risk classification for hiring systems; defensible audits are now part of the sales motion.
The Brief
What you'll do, and what you'll demonstrate.
Audit a hiring-screen classifier for cohort fairness with three standard metrics and produce both buyer-facing and internal remediation outputs.
Earning criteria — what you'll demonstrate
- Compute standard fairness metrics across cohorts
- Run counterfactual perturbation tests on a black-box classifier
- Translate audit findings into buyer-facing and internal documents
- Document audit methodology defensibly under regulatory scrutiny
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Trustworthy AI, Robustness, and Safety
Master · Responsible Ai
Strong alignment
This challenge maps to Trustworthy AI, Robustness, and Safety at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Fairness Evaluation
Apply fairness evaluation to solve real industry problems and demonstrate production-level capability.
- Disparate Impact
Apply disparate impact to solve real industry problems and demonstrate production-level capability.
- Audit Methodology
Apply audit methodology to solve real industry problems and demonstrate production-level capability.
- Counterfactual Testing
Apply counterfactual testing to solve real industry problems and demonstrate production-level capability.
- Python
Write clean, efficient Python for data processing, automation, and backend services.
- Risk Assessment
Apply risk assessment to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
AI Safety Researcher
Running a fairness audit with standard metrics and counterfactual tests is the AI safety researcher's textbook job at any regulated AI vendor.
This challenge sharpens
- fairness-evaluation
- audit-methodology
- counterfactual-testing
Data Scientist
Per-cohort metric work with honest CIs and a buyer-facing summary is senior data-science craft at HR-tech and finance-AI vendors.
This challenge sharpens
- fairness-evaluation
- disparate-impact
- audit-methodology
AI Product Manager
Owning the buyer-facing fairness story plus the internal remediation roadmap is increasingly part of the AI PM's job in regulated AI products.
This challenge sharpens
- audit-methodology
- risk-assessment
- fairness-evaluation