Audit a Sepsis Early-Warning Model for Subgroup Performance
Overview
What this challenge is about.
You receive a pre-trained vendor model, the training-data summary, and a held-out hospital-network evaluation set (about 18,000 ICU stays with sepsis labels). Compute AUROC + AUPRC, ECE + reliability diagrams, FPR/TPR gaps across the three protected attributes, and propose one mitigation for the largest gap. Then draft a drift-monitoring plan with thresholds, cadence, and escalation. Wrap into a 6-page committee-ready audit package.
The Brief
What you'll do, and what you'll demonstrate.
Run a defensible pre-deployment audit of a sepsis early-warning model with discrimination, calibration, fairness, and a monitoring plan.
Earning criteria — what you'll demonstrate
- Audit a clinical early-warning model across multiple axes
- Quantify subgroup gaps in a clinical-safety-relevant way
- Propose mitigations that respect clinical workflow constraints
- Communicate audit findings to a medical-informatics committee
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
AI Safety Researcher
Pre-deployment clinical audits paired with monitoring plans are the AI-safety-researcher's signature deliverable at consultancies serving healthcare networks.
This challenge sharpens
- fairness-metrics
- drift-detection
- model-monitoring
Applied AI Scientist
Producing committee-ready audit packages bridging technical evaluation and clinical governance is the applied-AI-scientist's craft at any healthtech consultancy.
This challenge sharpens
- model-calibration
- model-evaluation
- risk-stratification
MLOps Engineer
Designing post-deploy drift monitoring with clinical-action escalation paths is core MLOps work for any production clinical model.
This challenge sharpens
- drift-detection
- model-monitoring
- model-evaluation