Overview
What this challenge is about.
Pick a fusion architecture (early fusion via cross-attention, late fusion via score combination, or a unified multimodal encoder like FLAVA/CoCa). Train on the 14,000 pairs with a held-out clinic for evaluation. Report macro F1, per-category accuracy, and a deferral score (probability the model output should be reviewed by a clinician). Compare against text-only and image-only baselines. Write a 5-page clinical-handoff doc covering when the model is reliable, where deferral fires, and the known biases by patient skin tone (the dataset is partially imbalanced — be honest).
The Brief
What you'll do, and what you'll demonstrate.
Train a multimodal triage classifier with honest deferral behavior and write the clinical-handoff doc clinicians will rely on.
Earning criteria — what you'll demonstrate
- Apply multimodal fusion strategies to a real classification problem
- Calibrate probabilities for honest deferral decisions
- Audit a multimodal model for subgroup bias
- Communicate model limits clearly to a clinical audience
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
ML Researcher
Designing multimodal architectures, calibrating outputs, and auditing bias is the applied ML-research work that healthtech AI teams hire researchers to lead.
This challenge sharpens
- multimodal-fusion
- cross-attention
- calibration
Applied AI Scientist
Translating multimodal models into clinical-grade deferral systems is exactly the applied-AI work regulated healthtech teams need before deployment.
This challenge sharpens
- multimodal-fusion
- deferral-systems
- model-evaluation
AI Safety Researcher
Subgroup-bias auditing and deferral design are core AI safety work for any team shipping models into high-stakes domains.
This challenge sharpens
- calibration
- deferral-systems
- model-evaluation