The situation
The company (around USD 4M ARR, charges enterprises for an audit-grade plan UI on top of the OSS CLI) lost its second customer this quarter to a release that broke rollback on PostgreSQL 16.
Your task
Bring three critical modules of an untested OSS CLI from 8 percent to 75 percent meaningful test coverage and ship the work as reviewable PRs.
The brief
What you do
- Fork the repo (test mirror provided), set up Vitest with coverage and a CI workflow that fails under 75 percent on the targeted modules.
- Write characterization tests first (assert current behavior, including quirks) before adding new assertions.
- Find at least 4 bugs that show up only because the tests exist, file them as issues with a minimal failing test attached, and fix two of them in separate PRs.
You deliver
The PR series (link-only), a 2-page write-up covering tooling choices, bugs found, and the trade-off between coverage targets and meaningful tests, plus an updated CONTRIBUTING.md test section.
What you will practice
- Distinguish characterization tests from specification tests in a legacy codebase
- Configure a CI coverage gate that does not become a rubber stamp
- Write a minimal failing test for a real bug before fixing it
- Trade off coverage targets against test maintenance cost
A suggested way in
- Michael Feathers' 'Working Effectively with Legacy Code' is the playbook for characterization tests
- Mutation testing (Stryker) is a better signal than line coverage above 60 percent
- Keep PRs under 400 changed lines or reviewers will rubber-stamp them
Skills you will practice
Unit testing, Integration testing, Test coverage, Characterization testing, CI CD, Code review
How it is assessed
- Test meaningfulness30%
Tests assert behavior, not implementation details — survive a refactor.
- Coverage gate quality20%
CI gate is per-module and fails fast with a readable diff.
- Bug-find evidence25%
Each filed bug has a minimal failing test reproducing it.
- PR reviewability25%
PRs are scoped, sequenced, and pass review without major restructuring.
Resources
- VitestTool
Modern TypeScript-native test runner with built-in coverage.
- GitHub ActionsTool
Run the coverage gate on every PR.
- GitHubTool
Host the PR series and bug issues.
- StrykerTool
Mutation testing surfaces tests that look like coverage but assert nothing.
- Conventional CommitsTool
Keep PR titles scannable for reviewers and CHANGELOG.
- NotionTool
Author the 2-page write-up.
Finish it. Earn this.
A verified certificate with your name. Anyone can check it.
Time to practice.
Solve this challengeMore like this.
- Software EngineerBuild a Multi-Level Feedback Queue Scheduler in xv6
- Software EngineerReverse-Engineer a Stripped Binary to Restore Linking Symbols
- Software EngineerImplement a Distance-Vector Routing Algorithm
- AI EngineerEnd-to-End Lane Following on a Donkeycar Platform
-
Build a Multi-Level Feedback Queue Scheduler in xv6 Software Engineer Fork the xv6-riscv repository and implement MLFQ in proc.c with 3 priority queues, per-queue time slices (10ms / 20ms / 40ms), priority demotion on sl operating systemsschedulingc programming Open challenge → -
Reverse-Engineer a Stripped Binary to Restore Linking Symbols Software Engineer Receive the stripped shared object (Linux ELF, x86-64, ~280KB), a crash dump with 12 frames of indecipherable addresses, and 4 years of partial git hi reverse engineeringlinking and loadingmachine code Open challenge → -
Implement a Distance-Vector Routing Algorithm Software Engineer Implement distance-vector routing in Python: each simulated router maintains a distance vector, exchanges vectors with neighbors every 1 second of sim routingdistance vectornetwork simulation Open challenge → -
End-to-End Lane Following on a Donkeycar Platform AI Engineer Use the public Donkeycar Tub dataset (or collect about 30 minutes of driving on the simulator). Train a CNN-policy baseline (the Donkeycar default arc end to end learningimitation learningpytorch Open challenge →