Profile and Optimize a Virtual-Memory-Heavy Image Pipeline
Overview
What this challenge is about.
Receive the Go pipeline source, a representative batch (1,200 photos averaging 12MB each, with 30 outliers over 80MB), and host specs (4-core, 16GB RAM, Linux kernel 5.15). Run the batch under vmstat + sar + perf to capture page-fault rate, RSS trajectory, swap activity, and major vs. minor faults. Inspect /proc/PID/smaps to attribute RSS to file mmaps vs. anonymous allocations. Identify the top 3 causes of the 8GB spike (likely: large file mmaps without MADV_SEQUENTIAL, decoded-bitmap retention, COW from forked workers). Propose a 3-change plan with predicted RSS impact: add madvise hints, stream-decode instead of full-mmap for outliers, drop the fork-based worker pool. Deliver the measurement report, the proposed diffs, a prototype patch, the post-change measurement, and a 5-page write-up for the infrastructure team.
The Brief
What you'll do, and what you'll demonstrate.
Halve the 8GB peak RSS of an AVIF re-encoding pipeline using virtual-memory-aware changes, validated against the same batch.
Earning criteria — what you'll demonstrate
- Read /proc/PID/smaps to attribute RSS to specific mappings
- Distinguish major from minor page faults and their typical causes
- Apply madvise + streaming techniques to reduce peak memory
- Document the cause-and-effect chain clearly enough to fund the change
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Career mappings coming soon.