Migrate a Legacy Warehouse to a Lakehouse for an Edtech AI Platform
Overview
What this challenge is about.
Migrate a Legacy Warehouse to a Lakehouse for an Edtech AI Platform. Advanced challenge in code. Writing production code that solves real engineering problem...
The Brief
What you'll do, and what you'll demonstrate.
Prove that migrating one critical mart from Postgres to a lakehouse cuts runtime to under 30 minutes and lowers cost per query at the next growth tier.
This is not a coding exercise. It is the work a software engineer does between a Jira ticket and a merged PR. That distinction matters to every hiring manager who has seen candidates solve LeetCode problems and none who have shipped production code under real constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Model an open-table-format dataset (Delta or Iceberg) with appropriate partitioning
- Translate dbt-style transformations to a distributed engine
- Benchmark and explain analytical query performance fairly
- Plan a low-risk warehouse-to-lakehouse migration
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Data Engineering and Big Data Systems
Master · Data Engineering
Strong alignment
This challenge maps to Data Engineering and Big Data Systems at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Lakehouse Architecture
Apply lakehouse architecture to solve real industry problems and demonstrate production-level capability.
- Delta Lake
Apply delta lake to solve real industry problems and demonstrate production-level capability.
- Spark
Apply spark to solve real industry problems and demonstrate production-level capability.
- Dbt
Apply dbt to solve real industry problems and demonstrate production-level capability.
- Data Modeling
Apply data modeling to solve real industry problems and demonstrate production-level capability.
- Performance Benchmarking
Apply performance benchmarking to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Data Engineer
Lakehouse migrations are the dominant data-engineering project of the moment; shipping one PoC end-to-end is a strong portfolio piece.
This challenge sharpens
- lakehouse-architecture
- delta-lake
- spark
AI Solutions Architect
Writing a cost-and-risk migration plan a CTO can sign off on is core solutions-architect output.
This challenge sharpens
- data-modeling
- performance-benchmarking
- lakehouse-architecture
MLOps Engineer
Unblocking the morning retraining job is a quintessentially MLOps win; the lakehouse skills carry directly into MLOps platform work.
This challenge sharpens
- spark
- dbt
- performance-benchmarking