Overview
What this challenge is about.
Cost-Quality Prompt Optimization at Scale. Expert-level challenge in analysis. Analyzing real datasets and building models that drive decisions, earn a block...
The Brief
What you'll do, and what you'll demonstrate.
Find a Pareto-optimal prompt + model configuration that cuts spend by 40 percent on a 2M-call/week scoring pipeline without losing human-agreement quality.
This is not a data exercise. It is the work an analyst does when stakeholders need answers from messy data. That distinction matters to every hiring manager who has seen candidates describe statistical methods and none who have extracted insight from messy, real-world data.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Design a factorial prompt + model experiment under a fixed call budget
- Quantify the cost-quality trade-off rigorously (correlation, CIs, cost-per-call)
- Choose between prompt strategies (zero-shot, few-shot, CoT) on evidence
- Communicate optimization findings to an infrastructure-review audience
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Aligned coursework coming soon.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Prompt Optimization
Apply prompt optimization to solve real industry problems and demonstrate production-level capability.
- Cost Quality Tradeoff
Apply cost quality tradeoff to solve real industry problems and demonstrate production-level capability.
- Experiment Design
Apply experiment design to solve real industry problems and demonstrate production-level capability.
- Evaluation
Apply evaluation to solve real industry problems and demonstrate production-level capability.
- Structured Output
Apply structured output to solve real industry problems and demonstrate production-level capability.
- Ab Testing
Apply ab testing to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Prompt Engineer
Running structured prompt + model sweeps under a real production budget is exactly what senior prompt engineers do at AI-heavy companies.
This challenge sharpens
- prompt-optimization
- cost-quality-tradeoff
- experiment-design
MLOps Engineer
Owning the optimization + monitoring loop on a 2M-call/week pipeline is MLOps-engineer territory at any company spending serious money on LLM APIs.
This challenge sharpens
- cost-quality-tradeoff
- experiment-design
- evaluation
Applied AI Scientist
Factorial experiment design and rigorous cost-quality reporting is the rigor applied AI scientists bring to internal-tooling decisions.
This challenge sharpens
- experiment-design
- evaluation
- ab-testing