Build Hybrid Keyword-Plus-Embeddings Search for a Legal Document Portal
Overview
What this challenge is about.
Build Hybrid Keyword-Plus-Embeddings Search for a Legal Document Portal. Advanced challenge in design. Designing real products under real constraints, earn a...
The Brief
What you'll do, and what you'll demonstrate.
Design and prove a hybrid keyword-plus-embeddings retrieval system that measurably out-ranks the keyword-only baseline on a curated relevance set while remaining safe to ship to a live legal portal.
This is not a design exercise. It is the work a product designer does between a brief and a shipped interface. That distinction matters to every hiring manager who has seen candidates redesign Spotify's homepage and none who have worked under real product constraints.
When you finish, you will have something most graduates do not: a real-world deliverable, verified by Ewance, that you can show to a hiring manager and say "I did this. Here is the proof."
Earning criteria — what you'll demonstrate
- Combine lexical (BM25) and dense-embedding retrieval into a single ranked result using reciprocal rank fusion
- Select an embedding model by reasoning explicitly about indexing cost, latency, and retrieval quality trade-offs
- Evaluate retrieval quality with mean reciprocal rank and Recall@10, including confidence intervals and a labelled relevance set
- Assess relevance-judgment quality, including coverage of real user intents and agreement between annotators
- Design a low-risk production rollout using feature flags, telemetry, and an explicit rollback path
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Aligned coursework coming soon.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Information Retrieval
Apply information retrieval to solve real industry problems and demonstrate production-level capability.
- Bm25
Apply bm25 to solve real industry problems and demonstrate production-level capability.
- Vector Search
Apply vector search to solve real industry problems and demonstrate production-level capability.
- Hybrid Retrieval
Apply hybrid retrieval to solve real industry problems and demonstrate production-level capability.
- Relevance Evaluation
Apply relevance evaluation to solve real industry problems and demonstrate production-level capability.
- Feature Flags
Apply feature flags to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
Search / Retrieval Engineer
This challenge mirrors the core of a search-engineering role: standing up keyword and vector retrieval, fusing them, and proving quality gains on a labelled set. You leave able to defend an embedding-model choice and a relevance methodology to product and engineering stakeholders.
This challenge sharpens
- bm25
- vector-search
- hybrid-retrieval
Machine Learning Engineer (Search & Ranking)
Ranking-focused ML roles live or die by evaluation discipline. By building a relevance set, reporting mean reciprocal rank and Recall@10 with confidence intervals, and reasoning about embeddings, you practice the exact loop these teams run before shipping a model change.
This challenge sharpens
- relevance-evaluation
- vector-search
- information-retrieval
Backend Platform Engineer
Shipping a retrieval change to a 2.4-million-document production portal demands rollout discipline, not just model accuracy. The feature-flag staging, telemetry, and rollback work here maps directly to how platform engineers release risky changes safely.
This challenge sharpens
- feature-flags
- information-retrieval
- hybrid-retrieval