Pretrain a Small Vision Transformer with Self-Supervised Learning
Overview
What this challenge is about.
Self-supervised pretrain a ViT-Small on 80k histology tiles, fine-tune on 4k labeled ones, and compare baselines. Get a verifiable certificate.
The scenario
The startup (Series A, 35 people, partnered with two NHS pathology labs) is bottlenecked on annotation throughput; cutting labeled data needs by half would unlock two more product lines.
The Brief
What you'll do, and what you'll demonstrate.
Determine whether self-supervised pretraining on unlabeled tiles meaningfully outperforms ImageNet pretraining for this team's downstream histology task.
Earning criteria — what you'll demonstrate
- Implement a modern self-supervised pretraining objective end-to-end
- Design a fair fine-tuning comparison under a fixed compute budget
- Quantify model performance with statistical rigor (bootstrap CIs)
- Translate research findings into actionable team-process recommendations
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Advanced Deep Learning
Master · Deep Learning
Strong alignment
This challenge maps to Advanced Deep Learning at the Master level. It sharpens the same practical skills your coursework expects — but in a real industry context with actual constraints and deliverables.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
- Self Supervised Learning
Apply self supervised learning to solve real industry problems and demonstrate production-level capability.
- Vision Transformers
Apply vision transformers to solve real industry problems and demonstrate production-level capability.
- Pytorch
Apply pytorch to solve real industry problems and demonstrate production-level capability.
- Transfer Learning
Apply transfer learning to solve real industry problems and demonstrate production-level capability.
- Experiment Design
Apply experiment design to solve real industry problems and demonstrate production-level capability.
- Model Evaluation
Apply model evaluation to solve real industry problems and demonstrate production-level capability.
Careers
Career paths this challenge builds toward
Completing this challenge demonstrates skills that transfer directly to these roles:
ML Researcher
Implementing self-supervised pretraining on a real downstream task and writing it up with statistical rigor is precisely the workload of a first-year ML researcher at an applied lab.
This challenge sharpens
- self-supervised-learning
- vision-transformers
- experiment-design