Investigate Why Our Generative Model Memorizes Training Data
Overview
What this challenge is about.
Reproduce a Carlini-style extraction probe on a diffusion model, test three mitigations, and earn a verifiable certificate.
The scenario
The lab (anonymized; US-based, between Series B and Series C, around 200 employees, safety-focused) is preparing to open-source a small generative model and needs a defensible memorization story for the release announcement.
The Brief
What you'll do, and what you'll demonstrate.
Quantify how much training-data memorization a small open diffusion model exhibits and how well standard mitigations work.
Earning criteria — what you'll demonstrate
- Reproduce a published safety result on a real model
- Reason about the assumptions baked into extraction-attack methodologies
- Evaluate the cost/benefit of common memorization mitigations
- Communicate safety findings to a non-research policy audience
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Advanced Deep Learning
Master · Deep Learning
Strong alignment
This challenge maps to Advanced Deep Learning at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Generative Models
Apply generative models to solve real industry problems and demonstrate production-level capability.
- Memorization Analysis
Apply memorization analysis to solve real industry problems and demonstrate production-level capability.
- Differential Privacy
Apply differential privacy to solve real industry problems and demonstrate production-level capability.
- Evaluation
Apply evaluation to solve real industry problems and demonstrate production-level capability.
- Pytorch
Apply pytorch to solve real industry problems and demonstrate production-level capability.
- Safety Research
Apply safety research to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
ML Researcher
Threat-model articulation and assumption-tracking is the discipline that separates a citeable ML research project from a vibes-based one.
This challenge sharpens
- evaluation
- generative-models
- safety-research