Overview
What this challenge is about.
You receive a transcript dataset of 200 conversations (human-tagged with intent, slot values, and required outcome), a list of 8 supported intents, and tool stubs for 3 backend calls (balance, dispute file, dispute status). Design a dialogue manager combining intent classification, slot filling, and a state machine for multi-turn flows. Use an LLM for natural language, but keep the state machine explicit. Evaluate on (a) intent accuracy on 50 single-turn messages, (b) end-to-end task success on 30 multi-turn conversations (judged by whether the gold tool call was made with correct slots), and (c) appropriate-escalation precision/recall on 20 out-of-scope messages. Success is intent accuracy above 92 percent, task success above 80 percent, escalation F1 above 0.85.
The Brief
What you'll do, and what you'll demonstrate.
Build a multi-turn dialogue manager with explicit state tracking and escalation that meets accuracy and task-success targets.
Earning criteria — what you'll demonstrate
- Design a hybrid state-machine + LLM dialogue manager
- Implement and evaluate intent classification and slot filling
- Manage multi-turn context and graceful escalation
- Communicate dialogue-policy choices to a product team
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Career paths this builds toward
Canonical rolesNLP Engineer
Building a hybrid state-machine + LLM dialogue manager that meets task-success targets is the bread-and-butter NLP-engineer work at any consumer chatbot team.
This challenge sharpens
- dialogue-management
- intent-classification
- slot-filling
AI Engineer
Wiring tool calls into a dialogue policy with evaluation is core AI-engineer work in product orgs.
This challenge sharpens
- tool-use
- evaluation
- state-tracking
Machine Learning Engineer
Owning intent classifier quality plus end-to-end task evaluation is the discipline MLEs bring to conversational systems.
This challenge sharpens
- intent-classification
- evaluation
- state-tracking