Overview
What this challenge is about.
Take BERT-base (or DeBERTa-v3-base). Run layer-wise probes across at least 3 linguistic tasks: part-of-speech tagging, dependency arc classification, and semantic role labeling. Use the standard Probing Classifier paradigm with a frozen encoder + small MLP probe per task. Report accuracy by layer for each task; compare against a control task (e.g., random labels) to measure probe selectivity. Run 5 seeds. Write a 5-page workshop-style report with one striking layerwise plot per linguistic family.
The Brief
What you'll do, and what you'll demonstrate.
Run a layer-wise probing study on a pretrained encoder with selectivity controls and write the workshop report.
Earning criteria — what you'll demonstrate
- Implement layer-wise probing with proper controls
- Quantify what linguistic knowledge each transformer layer encodes
- Apply probe selectivity to avoid overclaiming
- Write a workshop-style interpretability report
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
ML Researcher
Layer-wise probing and selectivity analysis is the canonical interpretability work that ML researchers do on representation-learning teams.
This challenge sharpens
- interpretability
- probing
- transformers
Research Scientist
Following established protocols rigorously and reporting honestly with proper controls is the rigor expected of a junior research scientist on an interpretability team.
This challenge sharpens
- probing
- scientific-writing
- linguistic-evaluation
AI Safety Researcher
Interpretability skills bridge directly into AI safety research, where understanding what models know is foundational to alignment work.
This challenge sharpens
- interpretability
- probing
- scientific-writing