Overview
What this challenge is about.
Migrate a 200TB Data Lake from Parquet to Iceberg. Advanced challenge in design. Designing real products under real constraints, earn a blockchain-verified c...
The Brief
What you'll do, and what you'll demonstrate.
Migrate a 200TB Parquet data lake to Iceberg with no read regression and a runbook the data-platform team can execute table-by-table.
This is not a design exercise. It is the work a product designer does between a brief and a shipped interface. That distinction matters to every hiring manager who has seen candidates redesign Spotify's homepage and none who have worked under real product constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Choose the right Iceberg catalog for a multi-engine read pattern
- Design per-table migration strategies for varying scale
- Validate that schema evolution + time travel work end-to-end
- Produce a runbook the data platform team can execute table-by-table
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Big Data and Data-Intensive Systems
Master · Databases
Strong alignment
This challenge maps to Big Data and Data-Intensive Systems at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Iceberg
Apply iceberg to solve real industry problems and demonstrate production-level capability.
- Parquet
Apply parquet to solve real industry problems and demonstrate production-level capability.
- Data Lake
Apply data lake to solve real industry problems and demonstrate production-level capability.
- Spark
Apply spark to solve real industry problems and demonstrate production-level capability.
- Etl Pipelines
Apply etl pipelines to solve real industry problems and demonstrate production-level capability.
- Documentation
Apply documentation to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles: