Design Prompt Versioning and Observability for a Coding Assistant
Overview
What this challenge is about.
Design a prompt-registry schema, build a Python SDK, instrument LLM calls, scrub PII from logs, and create a Streamlit dashboard. Earn your verifiable certificate.
The scenario
The startup (~25 staff) wants prompt changes to feel as safe as code changes — versioned, observable, and rollback-able.
The Brief
What you'll do, and what you'll demonstrate.
Make prompt changes observable and versioned, with PII-safe response logging and a per-version metrics dashboard.
Earning criteria — what you'll demonstrate
- Design a prompt-registry schema with versions, owners, and environments
- Instrument an LLM-app for per-request observability
- Apply PII scrubbing to LLM request/response logs
- Build a metrics dashboard that catches silent regressions
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
LLM Application Development
Master · Ai Systems
Strong alignment
This challenge maps to LLM Application Development at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Prompt Versioning
Apply prompt versioning to solve real industry problems and demonstrate production-level capability.
- Observability
Apply observability to solve real industry problems and demonstrate production-level capability.
- Pii Scrubbing
Apply pii scrubbing to solve real industry problems and demonstrate production-level capability.
- Fastapi
Apply fastapi to solve real industry problems and demonstrate production-level capability.
- Streamlit
Apply streamlit to solve real industry problems and demonstrate production-level capability.
- Postgres
Apply postgres to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
MLOps Engineer
Prompt versioning and observability is core MLOps work for any LLM-powered product team.
This challenge sharpens
- prompt-versioning
- observability
- pii-scrubbing
AI Engineer
Instrumenting an LLM app for observability is what AI engineers ship next at any scaling AI startup.
This challenge sharpens
- fastapi
- observability
- prompt-versioning
Prompt Engineer
A prompt registry and a metrics dashboard are exactly the tooling prompt engineers need to do their job at production scale.
This challenge sharpens
- prompt-versioning
- observability
- pii-scrubbing