Diagnose Query Failures in an E-Commerce Search Box
Overview
What this challenge is about.
You receive 6 months of anonymized query logs (~480 million rows): query string, language hint, results-shown count, top-3 product clicks, and add-to-cart events. Build a notebook that finds zero-result and low-CTR queries, clusters them (embedding + HDBSCAN or simple TF-IDF clustering), tags each cluster with a dominant failure mode, and quantifies revenue impact by cluster. Deliver: notebook, a labeled cluster spreadsheet, and a 5-page diagnosis memo with prioritized fixes (typo tolerance, query expansion, synonym tables, language detection) and rough engineering-cost estimates.
The Brief
What you'll do, and what you'll demonstrate.
Find and quantify the dominant failure modes behind the failing 3% of search queries, with prioritized fixes.
Earning criteria — what you'll demonstrate
- Profile a real query log for failure modes
- Cluster queries and tag the dominant failure mechanism per cluster
- Quantify revenue impact of each failure cluster
- Translate IR failure analysis into prioritized engineering work
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Data Scientist
Query-log analysis with quantified revenue impact is the day-to-day of data scientists embedded with search and discovery teams.
This challenge sharpens
- query-log-analysis
- clustering
- revenue-impact-modeling
NLP Engineer
Translating failure-mode analysis into typo, synonym, and query-expansion work is what NLP engineers ship next at any e-commerce search team.
This challenge sharpens
- query-log-analysis
- ir-failure-analysis
- ir-evaluation
AI Product Manager
Owning the prioritized fix list with cost estimates is the AI PM's job on any search-and-discovery surface.
This challenge sharpens
- ir-failure-analysis
- revenue-impact-modeling
- clustering