Cost-Optimize an Embedding Pipeline for a Customer Support Knowledge Base
Overview
What this challenge is about.
You receive: (a) the current pipeline (full re-embed on any article change, OpenAI text-embedding-3-large, 3,072 dims) with one month of cost logs, (b) a sample of 5,000 articles with diffs across 30 days, and (c) a 200-query labeled benchmark from real tickets. Design at least two alternative strategies: e.g., switch to text-embedding-3-small + dimensionality reduction, or to an open-source encoder (bge-base or jina-embeddings-v2) plus content-hash-based change detection. Measure quality (recall@5 on the benchmark) and project the monthly cost across both alternatives. Recommend one path with a 90-day rollout plan. Success is at least 60% cost reduction with no more than 3 percentage points of recall@5 loss.
The Brief
What you'll do, and what you'll demonstrate.
Cut the embedding cost of a 90k-article daily-refresh knowledge base by at least 60% while losing no more than 3 percentage points of recall@5.
Earning criteria — what you'll demonstrate
- Compare embedding models on the quality/cost frontier for a real workload
- Apply content-hash change detection to avoid wasteful re-embedding
- Use Matryoshka-style dimensionality truncation to cut storage and similarity cost
- Translate ML measurement into a board-ready cost narrative
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Applied AI Scientist
Picking between embedding models on the quality/cost frontier and turning the result into a board memo is the day-to-day applied-AI-scientist deliverable inside any RAG-powered SaaS.
This challenge sharpens
- embedding-models
- evaluation
- cost-optimization
AI Engineer
Implementing change-detection and dimensionality truncation against a real ingestion pipeline is the practical engineering AI engineers do in customer-support and search products.
This challenge sharpens
- change-detection
- dimensionality-reduction
- rag
MLOps Engineer
The rollout plan with backfill + shadow comparison + rollback mirrors how MLOps engineers ship model swaps in production retrieval systems.
This challenge sharpens
- cost-optimization
- evaluation
- change-detection