Detect Fraudulent Refund Requests for a Mid-Market Marketplace
Overview
What this challenge is about.
You receive a labeled dataset with buyer history, seller history, shipping carrier, refund reason text, and outcome label (legit / fraud). Train and evaluate at least two classifiers, with a clear focus on calibration (predicted probabilities must actually mean what they say). Deliver an evaluation report showing the precision/recall trade-off at five threshold settings, plus a recommendation for the operating threshold that fits the team's current 200-cases-per-week manual-review capacity.
The Brief
What you'll do, and what you'll demonstrate.
Build a calibrated fraud-detection model whose operating point matches the trust & safety team's manual-review capacity.
Earning criteria — what you'll demonstrate
- Handle a heavily imbalanced classification problem with appropriate techniques
- Use probability calibration (Platt scaling, isotonic regression) and reliability diagrams
- Tie operating points to real operational constraints
- Communicate trade-offs to a non-technical operations team
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Data Scientist
Calibrated classifiers tied to operational capacity are exactly the kind of work that junior data scientists own at marketplaces, fintechs, and trust & safety teams.
This challenge sharpens
- classification
- model-calibration
- model-evaluation
Applied AI Scientist
Choosing operating points based on real ops capacity instead of pure metrics is what separates applied work from research and is a daily applied-AI-scientist task.
This challenge sharpens
- model-calibration
- imbalanced-classification
- model-evaluation
AI Engineer
Packaging a calibrated model with a defensible threshold recommendation is the kind of glue work AI engineers do when handing models to operations teams.
This challenge sharpens
- python
- feature-engineering
- model-calibration