Design a Distributed-Training Strategy for a Mid-Sized LLM
Overview
What this challenge is about.
Design a Distributed-Training Strategy for a Mid-Sized LLM. Expert-level challenge in research. Conducting rigorous research on real questions, earn a blockc...
The Brief
What you'll do, and what you'll demonstrate.
Pick and defend a parallelism strategy that hits a throughput and cost target for a 13B-parameter fine-tune on 32 GPUs.
This is not a research exercise. It is the work a researcher does to produce findings that withstand scrutiny. That distinction matters to every hiring manager who has seen candidates summarize papers and none who have produced original findings under expert review.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Pick a parallelism strategy (data/tensor/pipeline/hybrid) with quantitative justification
- Compute memory-per-GPU and tokens-per-second analytically
- Validate small-scale and extrapolate to production-scale
- Communicate distributed-training design choices to infrastructure leadership
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Aligned coursework coming soon.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Distributed Training
Apply distributed training to solve real industry problems and demonstrate production-level capability.
- Parallelism Strategies
Apply parallelism strategies to solve real industry problems and demonstrate production-level capability.
- Llm Training
Apply llm training to solve real industry problems and demonstrate production-level capability.
- Throughput Modeling
Apply throughput modeling to solve real industry problems and demonstrate production-level capability.
- Pytorch
Apply pytorch to solve real industry problems and demonstrate production-level capability.
- Cost Estimation
Apply cost estimation to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
ML Researcher
Choosing and defending a distributed-training strategy for an actual planned run is the daily reality of ML researchers at any LLM-training shop.
This challenge sharpens
- distributed-training
- llm-training
- throughput-modeling
Machine Learning Engineer
Memory and throughput modeling are exactly the skills MLEs use to keep large training runs from blowing up.
This challenge sharpens
- distributed-training
- throughput-modeling
- pytorch
MLOps Engineer
Cost-aware design at the 32-GPU scale is core MLOps work at any AI-research shop with a finite compute budget.
This challenge sharpens
- distributed-training
- cost-estimation
- parallelism-strategies