Overview
What this challenge is about.
You receive the model artifact (a gradient boosted regressor predicting expected annual loss per property), 2010-2020 training data, and a 2021-2024 holdout. Quantify covariate shift (using KS tests and energy distance on key features), prior shift (label-distribution differences), and concept drift (residual analysis over time). Identify which counties and feature groups are most affected. Deliver the analysis notebook, a per-region shift table, and a 4-page actuarial memo with a retrain-or-rebuild recommendation.
The Brief
What you'll do, and what you'll demonstrate.
Quantify covariate, prior, and concept drift on a 2010-2020-trained climate-risk model against 2021-2024 data and recommend retrain or rebuild.
Earning criteria — what you'll demonstrate
- Distinguish covariate, prior, and concept drift quantitatively
- Apply standard distribution-shift tests (KS, energy distance, MMD)
- Slice shift findings by region and feature group
- Communicate technical shift findings to a chief actuary
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Research Scientist
Quantifying distribution shift with rigorous statistical tests on a high-stakes actuarial model is the kind of work junior research scientists own at climate-risk shops.
This challenge sharpens
- distribution-shift
- covariate-shift
- concept-drift
AI Safety Researcher
Documenting drift evidence for a high-impact retrain-or-rebuild decision is exactly the AI safety researcher's contribution to safety-critical ML.
This challenge sharpens
- distribution-shift
- model-monitoring
- concept-drift
Data Scientist
Sliced shift analysis with actuarial-grade communication is a senior data-scientist responsibility in financial-services ML.
This challenge sharpens
- distribution-shift
- actuarial-analysis
- model-monitoring