Overview
What this challenge is about.
Profile a 24/7 LLM cluster, prototype two cost-optimization strategies, and validate SLA. Earn a verifiable certificate.
The scenario
The SF startup (around 50 staff, B2B AI agent platform) has 9 months of runway and views infra cost optimization as a runway-extension lever the CFO is watching.
The Brief
What you'll do, and what you'll demonstrate.
Cut LLM cluster cost by 30%+ via a prototyped optimization mix, without breaking the p99 latency SLA.
Earning criteria — what you'll demonstrate
- Profile real LLM-API usage to find cost-optimization levers
- Apply autoscaling, batching, and routing techniques to LLM serving
- Prove cost wins without breaking latency SLAs
- Translate engineering wins into a CFO-readable savings story
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
ML Engineering and Production ML
Master · Ai Systems
Strong alignment
This challenge maps to ML Engineering and Production ML at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Llm Serving
Apply llm serving to solve real industry problems and demonstrate production-level capability.
- Autoscaling
Apply autoscaling to solve real industry problems and demonstrate production-level capability.
- Ray
Apply ray to solve real industry problems and demonstrate production-level capability.
- Vllm
Apply vllm to solve real industry problems and demonstrate production-level capability.
- Cost Optimization
Apply cost optimization to solve real industry problems and demonstrate production-level capability.
- Kubernetes
Apply kubernetes to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
MLOps Engineer
Cost-optimizing LLM serving while holding SLAs is the platform-MLOps work that every AI startup eventually leans on once the cloud bill outgrows the COGS line.
This challenge sharpens
- llm-serving
- autoscaling
- cost-optimization
AI Engineer
Hands-on vLLM + Ray tuning is the AI-engineer skill set that startups hire for when they want one person to own model serving end to end.
This challenge sharpens
- vllm
- ray
- llm-serving
AI Solutions Architect
Designing the LLM serving topology and the cost-vs-SLA rollout plan is core AI solutions architecture work at any cloud provider or AI consultancy.
This challenge sharpens
- llm-serving
- kubernetes
- cost-optimization