Overview
What this challenge is about.
Build a canary rollout for a production recommender with auto-rollback using Prometheus + Grafana. Earn a verifiable certificate.
The scenario
The NYC startup (around 70 staff, social-discovery surface) experiments weekly and treats safe deploys as the unlock to ship faster.
The Brief
What you'll do, and what you'll demonstrate.
Ship a working canary-rollout system that auto-promotes good deploys and auto-rolls-back bad ones within 30 minutes.
Earning criteria — what you'll demonstrate
- Implement traffic-splitting in a production-grade serving stack
- Design statistically honest auto-promote/rollback rules
- Wire up the online metrics that matter (not just latency)
- Defend a rollout decision rule in writing to a skeptical reviewer
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
ML Engineering and Production ML
Master · Ai Systems
Strong alignment
This challenge maps to ML Engineering and Production ML at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Canary Deployment
Apply canary deployment to solve real industry problems and demonstrate production-level capability.
- Kubernetes
Apply kubernetes to solve real industry problems and demonstrate production-level capability.
- Ab Testing
Apply ab testing to solve real industry problems and demonstrate production-level capability.
- Prometheus
Apply prometheus to solve real industry problems and demonstrate production-level capability.
- Model Rollout
Apply model rollout to solve real industry problems and demonstrate production-level capability.
- Sequential Testing
Apply sequential testing to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
MLOps Engineer
Owning canary rollouts and auto-rollback logic is the work that defines a senior MLOps engineer on any team shipping models weekly.
This challenge sharpens
- canary-deployment
- model-rollout
- kubernetes
Machine Learning Engineer
MLEs who can ship safely (with traffic-split + sequential tests) move faster than ones who can only train; this challenge gives a portfolio piece for that skill.
This challenge sharpens
- model-rollout
- ab-testing
- sequential-testing
AI Solutions Architect
Designing rollout topologies and the decision rules that govern them is the architectural work AI solutions architects do for product teams.
This challenge sharpens
- canary-deployment
- kubernetes
- model-rollout