Design a Distributed Training Job for a 13B-Parameter Model
Overview
What this challenge is about.
Design a Distributed Training Job for a 13B-Parameter Model. Expert-level challenge in design. Designing real products under real constraints, earn a blockch...
The Brief
What you'll do, and what you'll demonstrate.
Pick and justify a distributed-training strategy for a 13B-param model on 32 H100s, validated on a smaller proxy, and write the runbook.
This is not a design exercise. It is the work a product designer does between a brief and a shipped interface. That distinction matters to every hiring manager who has seen candidates redesign Spotify's homepage and none who have worked under real product constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Choose between FSDP, TP, PP, and hybrid parallelism for a real workload
- Calculate per-GPU memory and throughput budgets from first principles
- Design fault-tolerant checkpoint cadence and storage
- Communicate distributed-training design to a non-systems audience
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Aligned coursework coming soon.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Distributed Training
Apply distributed training to solve real industry problems and demonstrate production-level capability.
- Fsdp
Apply fsdp to solve real industry problems and demonstrate production-level capability.
- Pytorch
Apply pytorch to solve real industry problems and demonstrate production-level capability.
- Gpu Systems
Apply gpu systems to solve real industry problems and demonstrate production-level capability.
- Checkpointing
Apply checkpointing to solve real industry problems and demonstrate production-level capability.
- Throughput Modeling
Apply throughput modeling to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Machine Learning Engineer
Designing a real distributed-training plan with throughput modeling and a runbook is the work that staff-track MLEs lead at any team training 10B+ models.
This challenge sharpens
- distributed-training
- fsdp
- throughput-modeling
AI Solutions Architect
Choosing parallelism strategies and writing the runbook that a client research team executes against is core AI solutions architecture work at cloud and consulting orgs.
This challenge sharpens
- distributed-training
- checkpointing
- gpu-systems
MLOps Engineer
Checkpoint cadence, failure recovery, and infrastructure-shaped training plans are the daily concerns of MLOps engineers on training-platform teams.
This challenge sharpens
- checkpointing
- gpu-systems
- pytorch