Overview
What this challenge is about.
Build a speaker diarization pipeline with face detection and speaker embeddings. Evaluate on a test set and earn your verifiable certificate.
The scenario
The Stockholm edtech (around 70 staff, Series A, 200,000 learners) treats per-speaker transcripts as the unlock for tutor-side analytics and learner review features.
The Brief
What you'll do, and what you'll demonstrate.
Cut Diarization Error Rate on tutoring sessions by fusing audio + video and prove the win on a 50-session held-out test.
Earning criteria — what you'll demonstrate
- Combine audio and video modalities at the right granularity
- Apply active-speaker detection to disambiguate similar voices
- Evaluate diarization with standard metrics (DER, JER)
- Hand off a multimodal pipeline to a non-ML platform team
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Multimodal Machine Learning
Master · Machine Learning
Strong alignment
This challenge maps to Multimodal Machine Learning at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Audio Visual Fusion
Apply audio visual fusion to solve real industry problems and demonstrate production-level capability.
- Speaker Diarization
Apply speaker diarization to solve real industry problems and demonstrate production-level capability.
- Active Speaker Detection
Apply active speaker detection to solve real industry problems and demonstrate production-level capability.
- Pytorch
Apply pytorch to solve real industry problems and demonstrate production-level capability.
- Pyannote
Apply pyannote to solve real industry problems and demonstrate production-level capability.
- Evaluation
Apply evaluation to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
ML Researcher
Fusing audio + video for diarization with honest DER evaluation is the applied multimodal research that edtech and conferencing AI teams hire for.
This challenge sharpens
- audio-visual-fusion
- speaker-diarization
- evaluation
Applied AI Scientist
Translating open research models into a working multimodal pipeline with a demo and handoff is core applied-AI-scientist work at AI-first startups.
This challenge sharpens
- audio-visual-fusion
- active-speaker-detection
- pytorch
Machine Learning Engineer
Shipping a production-shape AV pipeline that the platform team adopts is exactly the MLE work that edtech AI teams need on roadmap.
This challenge sharpens
- pytorch
- pyannote
- speaker-diarization