Design an Event-Driven Reporting Pipeline for an Enterprise BI Team
Overview
What this challenge is about.
Map the 9 source systems by extraction approach: CDC where possible (Debezium for SAP HANA + MES databases), event hooks where the source supports them (Salesforce streaming API), polling fallback (in-house apps). Design the Kafka topic layout (by source + entity), schema registry strategy (Avro + backward compat), and Snowflake target loading (Snowpipe + dbt). Prototype the SAP HANA CDC pipeline via Debezium into Kafka into Snowflake for one entity (likely Sales Order). Deliver design doc, prototype source + configs, and a target-state SLO sheet (latency, completeness).
The Brief
What you'll do, and what you'll demonstrate.
Design and prototype an event-driven reporting pipeline replacing 6-hour nightly batch loads from 9 source systems with sub-1-hour CDC-based ingestion into Snowflake.
Earning criteria — what you'll demonstrate
- Choose CDC vs event-hook vs polling per source system based on capability
- Design Kafka topic + schema strategy that survives source schema evolution
- Prototype an end-to-end CDC pipeline against a real enterprise source
- Set realistic latency + completeness SLOs for downstream analytics
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Career mappings coming soon.