Overview
What this challenge is about.
Pick a serving stack (Triton, Seldon Core, KServe, or BentoML). Implement two-model traffic splitting with a configurable percentage (start at 5%). Wire up online metric collection (latency, click-through rate, error rate) into Prometheus + Grafana. Define the decision rule with a sequential test (e.g., always-valid p-values via Always-Valid Inference) so you can decide early without inflating false positives. Demonstrate on a public dataset with a synthetic 'good' and 'bad' deploy; the system should auto-rollback the bad one within 30 minutes.
The Brief
What you'll do, and what you'll demonstrate.
Ship a working canary-rollout system that auto-promotes good deploys and auto-rolls-back bad ones within 30 minutes.
Earning criteria — what you'll demonstrate
- Implement traffic-splitting in a production-grade serving stack
- Design statistically honest auto-promote/rollback rules
- Wire up the online metrics that matter (not just latency)
- Defend a rollout decision rule in writing to a skeptical reviewer
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
MLOps Engineer
Owning canary rollouts and auto-rollback logic is the work that defines a senior MLOps engineer on any team shipping models weekly.
This challenge sharpens
- canary-deployment
- model-rollout
- kubernetes
Machine Learning Engineer
MLEs who can ship safely (with traffic-split + sequential tests) move faster than ones who can only train; this challenge gives a portfolio piece for that skill.
This challenge sharpens
- model-rollout
- ab-testing
- sequential-testing
AI Solutions Architect
Designing rollout topologies and the decision rules that govern them is the architectural work AI solutions architects do for product teams.
This challenge sharpens
- canary-deployment
- kubernetes
- model-rollout