Ewance
← All challenges Data Scientist

Interpretable-by-Design GAM for an Insurer's Claims Triage

Duration
2 weeks
Level
Intermediate
Type
Analysis
Team
1 to 5 people

The situation

The startup (around 50 people, working with mid-cap US insurers) competes on regulator-friendly modeling; an interpretable-by-design model can shave months from the carrier's internal review cycle even at a small accuracy cost.

Your task

Quantify the accuracy/interpretability trade-off between an Explainable Boosting Machine and a black-box LightGBM for claims-reserve estimation.

The brief

You start with

You receive an anonymized claims dataset (around 60,000 claims, target: log reserve), a feature schema (22 features), and an existing LightGBM baseline (held-out R^2 of 0.78).

What you do

  1. Train an Explainable Boosting Machine (EBM, a modern interpretable GAM) on the same features, tune via cross-validation, and produce per-feature shape plots.
  2. Compare EBM vs.
  3. LightGBM on held-out R^2, calibration by reserve decile, and actuarial sniff-test (you produce a per-feature shape-plot deck).
  4. Write a 2-page memo on the accuracy/interpretability trade-off and recommend a path forward.

What you will practice

  • Train and tune Explainable Boosting Machines as interpretable-by-design models
  • Compare interpretable vs. black-box models fairly on regression
  • Evaluate calibration alongside accuracy
  • Communicate model trade-offs to an actuarial audience

A suggested way in

  1. interpretml's EBM is the modern, scikit-learn-compatible GAM implementation
  2. Cross-validate both models to avoid bogus single-split conclusions
  3. Use decile calibration plots; insurers care more about tail calibration than overall R^2

Skills you will practice

Generalized additive models, Ebm, Interpretability, Model comparison, Calibration, Python

Where it leads

Data Scientist, AI Safety Researcher, Machine Learning Engineer

How it is assessed

  • EBM modeling quality30%

    EBM hits a defensible R^2 close to the LightGBM baseline.

  • Calibration analysis25%

    Per-decile calibration is reported with confidence intervals for both models.

  • Shape-plot interpretability25%

    Per-feature shape plots tell a coherent actuarial story.

  • Memo + recommendation20%

    Memo gives leadership a clear answer on which model to ship, with risks.

Resources

  • PythonTool

    Standard language for interpretml and modeling glue.

  • interpretTool

    Microsoft's interpretml library with EBM implementation and shape plots.

  • LightGBMTool

    Black-box baseline for the comparison.

  • PandasTool

    Process the claims data and per-decile calibration tables.

  • MatplotlibTool

    Render calibration curves and per-feature shape plots.

  • NotionTool

    Compose the 2-page accuracy-vs-interpretability memo.

Sample Ewance certificate of completion, verified by LearnCoin
Sample certificate
Your proof

Finish it. Earn this.

A verified certificate with your name. Anyone can check it.

Time to practice.

Solve this challenge
Related challenges

More like this.