Computer Science
Data Engineering & Pipelines Challenges
Data Engineering & Pipelines challenges put you inside the work of moving data reliably from source to insight. You'll develop skills in ETL Fundamentals, Data Pipeline Design, and Data Wrangling, and you'll write SQL for Analytics and dbt Models while building Airflow DAGs that orchestrate the flow.
From there you'll handle the harder edges — Kafka event streaming, Streaming-first design, Lakehouse architecture, and Data observability — working with Apache Spark and Snowflake or BigQuery query optimization the way data teams actually do. Each challenge you solve earns a verified credential you can share with recruiters.
- CodeIntermediateNew
Automate Retraining with a Drift-Triggered MLflow Pipeline
Stand up the pipeline end to end with the team's existing stack (MLflow tracking + model registry, Airflow orchestration). Wire Evidently to compute weekly drift; when drift cro…
- Mlflow
- Airflow Dags
- Data Drift Detection
ML Engineering and Production ML - AnalysisBeginnerNew
Build a Reproducible Pricing Analysis for a DTC Skincare Brand
You receive 24 months of order-line data (around 480,000 lines), a Shopify-style customer export, and a discount-code log. Build a Python pipeline that produces: SKU-level price…
- Data Wrangling
- Exploratory Data Analysis
- Cohort Analysis
Applied Data Analysis and Practical Data Science - DesignIntermediateNew
Design Schema Evolution for a Multi-Tenant Event Platform
Design a schema-evolution model covering: schema-registry topology (per-tenant subjects vs shared), compatibility modes per topic class (strict vs forward-only vs none), tenant-…
- Schema Evolution
- Kafka Event Streaming
- Schema Registry
Event-Driven Architecture - DesignIntermediateNew
Migrate a RabbitMQ Topology to Kafka for a Healthcare SaaS
Analyze the current RabbitMQ topology + 30 days of message volume data. Map each queue to a Kafka topic with the right partition key (likely patient_id, encounter_id, or clinic_…
- Kafka Event Streaming
- Rabbitmq
- Message Broker Migration
Event-Driven Architecture Practice your coursework on real scenarios.
Every challenge is shaped from real industry context — not generic exercises. The work mirrors what your degree prepares you for.
Why Ewance
- DesignBeginnerNew
Scaling a Sydney D2C Cosmetics Startup's Data Pipeline
You are tasked with designing a cloud-based data pipeline for GlowUp. The pipeline must ingest real-time user events (page views, purchases, returns) from web and mobile apps, p…
- Cloud Computing
- Apache Spark
- Nosql
Big Data and Cloud Technologies - DesignIntermediateNew
Design an Event-Driven Reporting Pipeline for an Enterprise BI Team
Map the 9 source systems by extraction approach: CDC where possible (Debezium for SAP HANA + MES databases), event hooks where the source supports them (Salesforce streaming API…
- Change Data Capture
- Kafka Event Streaming
- Debezium
Enterprise and Business Software Engineering - DesignIntermediateNew
Migrate a 200TB Data Lake from Parquet to Iceberg
Receive an inventory of the 200TB hot tier (around 1,200 tables, around 38 PB of historical data referenced), the current Spark + Trino read patterns, and 6 months of schema-cha…
- Iceberg
- Parquet
- Data Lake
Big Data and Data-Intensive Systems - DesignIntermediateNew
Event-Driven Architecture for an Order-to-Cash Pipeline
Design an event-driven architecture using a Kafka-like message bus (anonymized: assume a Kafka-compatible event log). Define: 9 event types with Avro schemas, partition strategy…
- Event Driven Architecture
- Kafka Event Streaming
- Saga Pattern
Software Architecture - Browse challenges
Explore role
Product Manager
Ship product that solves real user problems. Combine user research, prototyping, and stakeholder alignment to turn ambiguous briefs into measurable wins — the role at the centre of modern software teams.
- DesignIntermediateNew
Build a Scalable-System-Design Spec for a Streaming-Ingest Pipeline
Receive the current architecture (Kafka 3.6, Flink 1.18, ClickHouse 24.x), 4 weeks of production metrics (per-topic throughput, partition skew, Flink operator backpressure, Clic…
- Scalable System Design
- Kafka Event Streaming
- Flink
Performance Engineering of Software Systems - AnalysisBeginnerNew
Sales Performance Analysis for a 40-Person SaaS Scale-Up
You will receive a dataset containing 500+ sales opportunities with fields like deal value, stage, source, close date, and account size. Your challenge is to design a data mart …
- Data Warehousing
- Etl Fundamentals
- Olap
Business Intelligence - StrategyIntermediateNew
Designing a BI Strategy for a Regional Retail Chain
Your challenge is to design a BI architecture including a data warehouse (conceptual and logical models), an ETL strategy to integrate data from multiple sources (POS, inventory…
- Data Warehousing
- Etl Fundamentals
- Olap
Business Intelligence - CodeIntermediateNew
Build a Feature Store Backbone for a Healthtech ML Team
You receive synthetic wearable telemetry (heart rate, accelerometer, sleep stages) for around 5,000 patients across 90 days, plus the existing scattered feature scripts from the…
- Feature Engineering
- Data Modeling
- Python Or Javascript
Data Engineering and Big Data Systems Build a verifiable portfolio.
Submissions become evidence. Reviewers with shipping experience score against a rubric; the result becomes a credential anyone can verify.
Why Ewance
- CodeIntermediateNew
Reproducible Patient-Cohort Analysis for a Pharma AI Vendor
You receive a written cohort definition (type-2 diabetes patients on metformin for at least 90 days, aged 40-70) and a target output: 12-month HbA1c change distribution plus a K…
- Reproducible Analysis
- Cohort Analysis
- Survival Analysis
Applied Data Analysis and Practical Data Science - CodeIntermediateNew
Design a Change-Data-Capture Pipeline for an E-Commerce Reseller
Receive the MySQL schema (220 tables), 7 days of binlog samples, and the data team's freshness + correctness requirements. Design the CDC pipeline: Debezium for MySQL binlog cap…
- Change Data Capture
- Debezium
- Kafka Event Streaming
Big Data and Data-Intensive Systems - CodeIntermediateNew
Build a Streaming Pipeline for Real-Time Fraud Detection
Receive 30 days of anonymized card-transaction events (around 240M events total), the team's existing batch features (cardholder behavior summaries), and a pre-trained fraud-sco…
- Stream Processing
- Kafka Event Streaming
- Flink
Big Data and Data-Intensive Systems - DesignIntermediateNew
Stand Up a Feature Store for a Series-B Fintech
Pick one priority feature group (recommend the 25 transaction-history features used by the fraud model). Define the offline source-of-truth (likely Snowflake or BigQuery), the o…
- Feature Store
- Feature Engineering
- Airflow Dags
ML Engineering and Production ML - DesignIntermediateNew
Design a Real-Time Order Pipeline for a Fintech Payments Platform
You receive a synthetic Kafka stream of around 500 transactions per second, a static merchant dimension table (about 80,000 rows), and a daily FX rate snapshot. Design an end-to…
- Streaming Data
- Kafka Event Streaming
- Stream Processing
Data Engineering and Big Data Systems - AnalysisIntermediateNew
Frequent-Itemset Mining on a Grocery Retailer's Basket History
Load 18 months of basket-level transaction data (Parquet, around 92 GB) into a Spark cluster. Run FP-growth at support thresholds tuned per category (food vs household vs fresh)…
- Frequent Itemset Mining
- Fp Growth
- Apache Spark
Data Mining and Information Retrieval - DesignIntermediateNew
Build a Feature Store for a Fintech Fraud Team
You will design a feature-store layer covering 12 representative fraud features (account-level, merchant-level, transaction-level), with both batch (Spark) and online (low-laten…
- Feature Stores
- Data Pipelines
- Apache Spark
Machine Learning at Scale - CodeIntermediateNew
Instrument Network Telemetry for an ISP's Backbone
Receive the backbone topology (12 routers across 4 PoPs, mix of Cisco IOS XR + Juniper Junos), the current SNMP-based monitoring stack, and 4 weeks of customer-complaint tickets…
- Network Telemetry
- Gnmi
- Kafka Event Streaming
Advanced Computer Networks - CodeIntermediateNew
Migrate a Legacy Warehouse to a Lakehouse for an Edtech AI Platform
You receive a Postgres dump of around 50 GB and the current dbt models that produce the student-attempts mart. Land the raw data in object storage (S3 or GCS) as Parquet partiti…
- Lakehouse Architecture
- Delta Lake
- Apache Spark
Data Engineering and Big Data Systems - AnalysisBeginnerNew
Build a Public Open-Data Dashboard for Urban Mobility
Pull the city's open-data cyclist-collision dataset (10 years of incidents, geocoded). Define a clear before/after window around the protected-lane rollout, control for traffic-…
- Exploratory Data Analysis
- Data Wrangling
- Geospatial Analysis
Applied Data Analysis and Practical Data Science - AnalysisBeginnerNew
Audit a Climate-Tech Sensor Dataset for Production Readiness
You receive 18 months of raw sensor readings from 1,200 sensors (about 800M rows), plus a sensor-metadata table (location, firmware version, deployment date). Profile the data f…
- Data Quality Audit
- Data Profiling
- Time Series Analysis
Applied Data Analysis and Practical Data Science - DesignSeniorNew
Designing a Data Warehouse for a Renewable Energy Firm
You are given sample data from three sources: energy production logs, weather data, and equipment maintenance records. Your task is to: (1) design a star schema with fact and di…
- Data Warehousing
- Star Schema
- Etl Fundamentals
Database Systems
How it works
From brief to credential, in six steps.
Step 01
Browse challenges aligned to your studies.
Step 02
Accept the one that fits your goals.
Step 03
Work through it with AI Copilot guidance.
Step 04
Submit for structured evaluation.
Step 05
Earn a verified credential.
Step 06
Add it to LinkedIn with one click.
Related skill families
Browse all skillsIndustry teams behind a decade of practitioner briefs
Hiring from this pool?
Sponsor a challenge and meet candidates through actual work.
Industry teams can shape briefs around the skills they hire for, then evaluate students on rubric-scored deliverables — not resumes.