Overview
What this challenge is about.
Implement PageRank with BSP in Spark or MPI, validate scores, and benchmark scaling across nodes to earn your verifiable certificate.
The scenario
The lab is a university-affiliated search-and-recommendations research group of about 25 people that maintains a 1.5-billion-edge web graph for ranking experiments in partnership with three web publishers. It has researchers and engineers but no dedicated infrastructure or operations team, so any new pipeline must be reproducible by non-specialists.
The Brief
What you'll do, and what you'll demonstrate.
Implement and benchmark Bulk Synchronous Parallel PageRank on the lab's web graph across a 16-node cluster, with correctness validation and an honest weak-scaling efficiency report.
Earning criteria — what you'll demonstrate
- Implement PageRank under a Bulk Synchronous Parallel computation model using a message-passing graph framework.
- Measure and reason about weak-scaling efficiency and per-superstep communication cost on a multi-node cluster.
- Validate distributed graph-algorithm correctness against a reference with known ground-truth values.
- Make a reproducibility-first infrastructure recommendation appropriate for a team without operations staff.
- Diagnose where communication, rather than computation, dominates distributed graph workloads.
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Aligned coursework coming soon.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Parallel Algorithms
Apply parallel algorithms to solve real industry problems and demonstrate production-level capability.
- Bsp
Apply bsp to solve real industry problems and demonstrate production-level capability.
- Graph Algorithms
Apply graph algorithms to solve real industry problems and demonstrate production-level capability.
- Mpi
Apply mpi to solve real industry problems and demonstrate production-level capability.
- Distributed Systems
Apply distributed systems to solve real industry problems and demonstrate production-level capability.
- Benchmarking
Apply benchmarking to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Distributed Systems Engineer
Building and benchmarking a BSP graph computation across a multi-node cluster mirrors the core work of distributed systems engineering: reasoning about synchronization barriers, communication cost, and scaling efficiency under real hardware constraints.
This challenge sharpens
- bsp
- distributed-systems
- benchmarking
Backend / Data Infrastructure Engineer
Turning a 14-hour single-machine job into a reproducible cluster pipeline is exactly the data-infrastructure mandate — owning graph workloads, partitioning, and one-command reproducibility for teams without dedicated operations staff.
This challenge sharpens
- graph-algorithms
- distributed-systems
- parallel-algorithms
High-Performance Computing Engineer
Implementing PageRank in MPI and characterizing weak-scaling and communication bottlenecks builds the HPC skill set: parallel algorithm design, cluster benchmarking, and honest performance reporting on tightly coupled workloads.
This challenge sharpens
- parallel-algorithms
- mpi
- benchmarking