Build a Vector-Search Backend for an Enterprise AI Knowledge Assistant
Overview
What this challenge is about.
Build a hybrid retrieval API for an enterprise AI knowledge assistant. Parse 20,000 PDFs with OCR and semantic chunking. Earn a verifiable certificate.
The scenario
The startup (around 35 staff, post-seed, paid pilot with one Fortune-500 industrial customer) needs the retrieval layer to be production-grade before the model team builds the response-generation features on top.
The Brief
What you'll do, and what you'll demonstrate.
Build a RAG ingest-and-retrieval backend that hits recall-at-10 above 0.85 and p95 latency under 300 ms on an enterprise PDF corpus.
Earning criteria — what you'll demonstrate
- Design a chunking strategy informed by retrieval evaluation
- Operate an embedding pipeline at corpus scale
- Combine vector and lexical retrieval into a hybrid system
- Measure retrieval quality with standard metrics (recall@k, MRR)
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Data Engineering and Big Data Systems
Master · Data Engineering
Strong alignment
This challenge maps to Data Engineering and Big Data Systems at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Rag
Apply rag to solve real industry problems and demonstrate production-level capability.
- Vector Search
Apply vector search to solve real industry problems and demonstrate production-level capability.
- Embeddings
Apply embeddings to solve real industry problems and demonstrate production-level capability.
- Document Parsing
Apply document parsing to solve real industry problems and demonstrate production-level capability.
- Python
Write clean, efficient Python for data processing, automation, and backend services.
- Retrieval Evaluation
Apply retrieval evaluation to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
AI Engineer
Building production-grade RAG retrieval backends is the single most common AI-engineer job description right now; this challenge ships the load-bearing piece.
This challenge sharpens
- rag
- vector-search
- embeddings
Data Engineer
Corpus-scale ingest with parsing fallbacks and resumability is core data-engineering work that supports any RAG or search team.
This challenge sharpens
- document-parsing
- python
- retrieval-evaluation
Machine Learning Engineer
Owning the retrieval-evaluation harness with recall@k and MRR mirrors how MLEs run model evals at scale.
This challenge sharpens
- retrieval-evaluation
- embeddings
- vector-search