Overview
What this challenge is about.
Build a FastAPI streaming chat endpoint with Redis caching and LLM fallbacks. Write pytest tests. Get a verifiable certificate.
The scenario
The startup (~30 staff, ~USD 4M ARR) has had two customer-visible outages tied to upstream provider rate limits in the last quarter.
The Brief
What you'll do, and what you'll demonstrate.
Ship a streaming, cached, fallback-capable RAG endpoint with per-request cost tracking and a clear runbook.
Earning criteria — what you'll demonstrate
- Implement Server-Sent Event streaming for LLM responses
- Design a multi-provider fallback chain
- Cache LLM responses safely with the right key composition
- Track and attribute per-request LLM cost
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
LLM Application Development
Master · Ai Systems
Strong alignment
This challenge maps to LLM Application Development at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Llm Api Integration
Apply llm api integration to solve real industry problems and demonstrate production-level capability.
- Streaming
Apply streaming to solve real industry problems and demonstrate production-level capability.
- Response Caching
Apply response caching to solve real industry problems and demonstrate production-level capability.
- Fallback Design
Apply fallback design to solve real industry problems and demonstrate production-level capability.
- Cost Tracking
Apply cost tracking to solve real industry problems and demonstrate production-level capability.
- Fastapi
Apply fastapi to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
AI Engineer
Shipping a production LLM endpoint with streaming, caching, and fallback is the day-one work of AI engineers at any AI-product startup.
This challenge sharpens
- llm-api-integration
- streaming
- fallback-design
MLOps Engineer
Owning the cost-tracking schema and the runbook bridges directly into MLOps work on inference platforms.
This challenge sharpens
- cost-tracking
- fallback-design
- response-caching
Machine Learning Engineer
The fallback chain plus the local open-weight model brings together application engineering and ML deployment in one project.
This challenge sharpens
- fallback-design
- llm-api-integration
- fastapi