Build a Fairness Evaluation Harness for a Credit-Score Model
Overview
What this challenge is about.
Build a fairness evaluation harness for a credit-score model in Python and produce a report. End with a verifiable certificate.
The scenario
The fintech (around 200 staff, around 400,000 active borrowers across Singapore and Malaysia) is preparing for a Monetary Authority of Singapore (MAS) fairness review and needs every model release to come with a standard fairness audit.
The Brief
What you'll do, and what you'll demonstrate.
Build a reusable fairness evaluation harness with multiple group metrics and bootstrap intervals, plus a release-ready evaluation report.
Earning criteria — what you'll demonstrate
- Implement multiple group-fairness metrics from first principles
- Apply bootstrap methods for honest confidence intervals
- Reason about intersecting protected attributes
- Communicate fairness results to a risk-team audience
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
AI Measurement and Evaluation
Master · Responsible Ai
Strong alignment
This challenge maps to AI Measurement and Evaluation at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Algorithmic Fairness
Apply algorithmic fairness to solve real industry problems and demonstrate production-level capability.
- Statistical Evaluation
Apply statistical evaluation to solve real industry problems and demonstrate production-level capability.
- Python
Write clean, efficient Python for data processing, automation, and backend services.
- Bootstrap Methods
Apply bootstrap methods to solve real industry problems and demonstrate production-level capability.
- Model Evaluation
Apply model evaluation to solve real industry problems and demonstrate production-level capability.
- Test Driven Development
Apply test driven development to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Data Scientist
Shipping a reusable fairness harness with proper statistics is the data scientist's contribution to any regulated lending model release.
This challenge sharpens
- algorithmic-fairness
- statistical-evaluation
- model-evaluation
Machine Learning Engineer
Test-driven, edge-case-aware code is the MLE's craft when productionizing evaluation infra.
This challenge sharpens
- python
- test-driven-development
- model-evaluation
AI Safety Researcher
Group fairness and intersectional analysis sit squarely in the safety researcher's responsible-AI portfolio.
This challenge sharpens
- algorithmic-fairness
- statistical-evaluation
- bootstrap-methods