Optimize Hyperparameters with Bayesian Optimization on a Tight Budget
Overview
What this challenge is about.
You receive a B2B-SaaS churn dataset (about 12,000 customer-month rows, 38 features) and a fixed sweep budget of 40 trials per model family. Implement a Bayesian optimizer (Optuna's TPE or Ax's GPEI) and compare against random search on 3 model families: lightgbm, a small MLP (multi-layer perceptron) in PyTorch, and a balanced random-forest. Track validation AUC vs. cumulative compute spent. Recommend the optimizer + model family combination the team should standardize on.
The Brief
What you'll do, and what you'll demonstrate.
Pick a hyperparameter-optimization strategy that reaches a target validation AUC in the fewest trials across multiple model families.
Earning criteria — what you'll demonstrate
- Apply Bayesian hyperparameter optimization in practice
- Compare optimization strategies under a fixed compute budget
- Reason about exploration vs. exploitation in TPE-style optimizers
- Communicate a cost-quality trade-off to a non-ML founder
Program Fit
Where this fits in your program.
Sharpens the same skills your degree expects you to demonstrate.
Skills
Skills you'll demonstrate.
Each one shows up on your verified credential.
Careers
Roles this prepares you for.
Real titles. Real skill bridges. Pick the one closest to your trajectory.
Machine Learning Engineer
Owning a hyperparameter-optimization upgrade with measurable cost savings is the kind of high-leverage MLE project early-stage startups remember at promotion time.
This challenge sharpens
- hyperparameter-tuning
- optuna
- ensemble-methods
Data Scientist
Cross-family model comparison with honest convergence diagnostics is daily data-science work at any analytics team.
This challenge sharpens
- bayesian-optimization
- model-evaluation
- ensemble-methods