Build a Serverless ETL Pipeline for a Climate-Tech Sensor Fleet
Overview
What this challenge is about.
Build a Serverless ETL Pipeline for a Climate-Tech Sensor Fleet. Advanced challenge in code. Writing production code that solves real engineering problems, e...
The Brief
What you'll do, and what you'll demonstrate.
Replace a fragile cron pipeline with a serverless, idempotent ETL that survives single-region cloud incidents and ingests 50M rows/day reliably.
This is not a coding exercise. It is the work a software engineer does between a Jira ticket and a merged PR. That distinction matters to every hiring manager who has seen candidates solve LeetCode problems and none who have shipped production code under real constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Compose a real serverless ETL pipeline using managed cloud primitives
- Apply idempotency and reprocessing patterns
- Validate and quarantine bad data at ingest
- Write an Architecture Decision Record that survives executive review
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Cloud Computing for Data and ML
Master · Data Engineering
Strong alignment
This challenge maps to Cloud Computing for Data and ML at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Serverless Architecture
Apply serverless architecture to solve real industry problems and demonstrate production-level capability.
- Etl Pipelines
Apply etl pipelines to solve real industry problems and demonstrate production-level capability.
- Infrastructure As Code
Apply infrastructure as code to solve real industry problems and demonstrate production-level capability.
- Data Validation
Apply data validation to solve real industry problems and demonstrate production-level capability.
- Cloud Services
Apply cloud services to solve real industry problems and demonstrate production-level capability.
- Monitoring Design
Apply monitoring design to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Data Engineer
Serverless ETL on managed cloud primitives with infrastructure-as-code is the day-one work of a junior data engineer at any IoT or climate-tech company.
This challenge sharpens
- etl-pipelines
- serverless-architecture
- data-validation
MLOps Engineer
Idempotent pipelines + infrastructure-as-code + monitoring habits transfer directly into MLOps work on production model pipelines.
This challenge sharpens
- infrastructure-as-code
- monitoring-design
- cloud-services
AI Solutions Architect
Writing an Architecture Decision Record that survives executive review is the core craft of an AI solutions architect.
This challenge sharpens
- serverless-architecture
- cloud-services
- infrastructure-as-code