Overview
What this challenge is about.
You receive 300 real invoice transcripts (already OCR-ed) labeled with 14 target fields, plus the current production prompt and its 12 percent failure log. Design a new prompt using JSON-mode and a strict JSON Schema (anyOf for optional fields, regex for IBAN, ISO 4217 for currency). Add 3-5 few-shot examples picked specifically from the failure log. Evaluate on a held-out 60-invoice test set on (a) JSON validity, (b) per-field exact-match accuracy, and (c) total-amount numerical match. Success is 100 percent JSON validity, per-field accuracy above 92 percent on average, and total-amount match above 98 percent.
The Brief
What you'll do, and what you'll demonstrate.
Redesign an invoice-extraction prompt with JSON-mode + Schema + targeted few-shot examples to meet a sub-2 percent failure SLA.
Earning criteria — what you'll demonstrate
- Use JSON-mode and JSON Schema to enforce structured LLM outputs
- Mine real failure logs to curate few-shot examples that target known failure modes
- Design an evaluation harness for extraction quality with multiple metrics
- Iterate prompt design under a production SLA
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Prompt Engineer
Designing a structured-output prompt that meets a production SLA on real customer data is the day-one job of a prompt engineer at any AI product startup.
This challenge sharpens
- structured-output
- json-schema
- few-shot-prompting
AI Engineer
Building the prompt-plus-eval-harness boundary between the model and the product is the AI-engineer skillset hiring managers screen for.
This challenge sharpens
- prompt-evaluation
- python
- failure-analysis
NLP Engineer
Information extraction with field-level accuracy and structured outputs is core NLP-engineer work in B2B AI.
This challenge sharpens
- structured-output
- prompt-evaluation
- failure-analysis