Ewance
← All challenges Data Scientist

Explain a Credit-Risk Model with SHAP for a Fintech

Duration
2 weeks
Level
Intermediate
Type
Analysis
Team
1 to 5 people

The situation

The fintech (around 350 people, around USD 4 billion in annual transaction volume) is regulated by the Monetary Authority of Singapore (MAS) and must show explainability and fairness diligence on any model touching credit before production roll-out.

Your task

Produce a SHAP-based explainability + fairness toolkit for a credit-risk model that satisfies the compliance team's model-risk requirements.

The brief

You start with

You receive a trained XGBoost credit-risk model (binary default prediction), the training feature schema (38 features), and a held-out 10,000-sample test set with labels.

What you do

  1. Compute global SHAP feature-importance plots, beeswarm + dependence plots, and per-decision local explanations for the test set.
  2. Run a fairness slice on default-rate predictions across 6 SME industry categories (services, retail, manufacturing, logistics, hospitality, agri).
  3. Produce a one-page explanation card template a credit officer can paste into the review file.
  4. Write a 2-page compliance memo with caveats on SHAP's limitations.

What you will practice

  • Apply SHAP for global and local model explanation on tabular data
  • Run a fairness slice analysis across protected-ish subpopulations
  • Translate explanation outputs into operator-friendly artifacts
  • Communicate XAI limitations to a compliance audience

A suggested way in

  1. Use TreeSHAP via the shap library for fast exact SHAP values on XGBoost
  2. Be explicit about base value vs. raw probability in local explanations; this trips up reviewers
  3. Report both predicted default rate AND calibration per slice; SHAP doesn't catch miscalibration

Skills you will practice

Shap, Interpretability, Fairness analysis, Model validation, Xgboost, Python

Where it leads

Data Scientist, AI Safety Researcher, Machine Learning Engineer

How it is assessed

  • Explanation quality30%

    Global and local SHAP outputs are correct and clearly visualized.

  • Fairness slice analysis25%

    Per-industry default-rate and calibration are reported with confidence intervals.

  • Operator-card design20%

    Explanation card is usable by a credit officer in under 60 seconds.

  • Compliance memo25%

    Memo addresses SHAP's known limitations (correlated features, off-manifold inputs) honestly.

Resources

  • PythonTool

    Standard language for SHAP and downstream analysis.

  • shapTool

    Reference library for SHAP values with TreeSHAP for XGBoost.

  • PandasTool

    Slice the test set by industry and compute fairness metrics.

  • MatplotlibTool

    Render dependence plots and per-slice calibration curves.

  • FigmaTool

    Design the one-page explanation card template.

  • NotionTool

    Author the 2-page compliance memo.

Sample Ewance certificate of completion, verified by LearnCoin
Sample certificate
Your proof

Finish it. Earn this.

A verified certificate with your name. Anyone can check it.

Time to practice.

Solve this challenge
Related challenges

More like this.