Design Parallel I/O for a Climate-Simulation Data Pipeline
Overview
What this challenge is about.
Design Parallel I/O for a Climate-Simulation Data Pipeline. Advanced challenge in design. Designing real products under real constraints, earn a blockchain-v...
The Brief
What you'll do, and what you'll demonstrate.
Replace per-rank serial HDF5 output with parallel HDF5 + MPI-IO collective writes, cutting I/O time to under 15 percent of wall-clock at 512 ranks.
This is not a design exercise. It is the work a product designer does between a brief and a shipped interface. That distinction matters to every hiring manager who has seen candidates redesign Spotify's homepage and none who have worked under real product constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Replace per-rank file output with collective parallel I/O
- Align HDF5 chunk size to Lustre stripe size for max bandwidth
- Use MPI-IO collective writes to amortize metadata cost
- Benchmark parallel I/O at scale and identify the binding constraint
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
High-Performance and Scientific Computing
Master · Systems
Strong alignment
This challenge maps to High-Performance and Scientific Computing at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Mpi Io
Apply mpi io to solve real industry problems and demonstrate production-level capability.
- Parallel Hdf5
Apply parallel hdf5 to solve real industry problems and demonstrate production-level capability.
- Lustre Filesystem
Apply lustre filesystem to solve real industry problems and demonstrate production-level capability.
- Fortran Programming
Apply fortran programming to solve real industry problems and demonstrate production-level capability.
- Chunking Strategy
Apply chunking strategy to solve real industry problems and demonstrate production-level capability.
- Performance Analysis
Apply performance analysis to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles: