Ewance
← All challenges Software Engineer

Add a Test Harness to an Untested Open-Source CLI

Duration
2 weeks
Level
Intermediate
Type
Code
Team
1 to 5 people

The situation

The company (around USD 4M ARR, charges enterprises for an audit-grade plan UI on top of the OSS CLI) lost its second customer this quarter to a release that broke rollback on PostgreSQL 16.

Your task

Bring three critical modules of an untested OSS CLI from 8 percent to 75 percent meaningful test coverage and ship the work as reviewable PRs.

The brief

What you do

  1. Fork the repo (test mirror provided), set up Vitest with coverage and a CI workflow that fails under 75 percent on the targeted modules.
  2. Write characterization tests first (assert current behavior, including quirks) before adding new assertions.
  3. Find at least 4 bugs that show up only because the tests exist, file them as issues with a minimal failing test attached, and fix two of them in separate PRs.

You deliver

The PR series (link-only), a 2-page write-up covering tooling choices, bugs found, and the trade-off between coverage targets and meaningful tests, plus an updated CONTRIBUTING.md test section.

What you will practice

  • Distinguish characterization tests from specification tests in a legacy codebase
  • Configure a CI coverage gate that does not become a rubber stamp
  • Write a minimal failing test for a real bug before fixing it
  • Trade off coverage targets against test maintenance cost

A suggested way in

  1. Michael Feathers' 'Working Effectively with Legacy Code' is the playbook for characterization tests
  2. Mutation testing (Stryker) is a better signal than line coverage above 60 percent
  3. Keep PRs under 400 changed lines or reviewers will rubber-stamp them

Skills you will practice

Unit testing, Integration testing, Test coverage, Characterization testing, CI CD, Code review

How it is assessed

  • Test meaningfulness30%

    Tests assert behavior, not implementation details — survive a refactor.

  • Coverage gate quality20%

    CI gate is per-module and fails fast with a readable diff.

  • Bug-find evidence25%

    Each filed bug has a minimal failing test reproducing it.

  • PR reviewability25%

    PRs are scoped, sequenced, and pass review without major restructuring.

Resources

  • VitestTool

    Modern TypeScript-native test runner with built-in coverage.

  • GitHub ActionsTool

    Run the coverage gate on every PR.

  • GitHubTool

    Host the PR series and bug issues.

  • StrykerTool

    Mutation testing surfaces tests that look like coverage but assert nothing.

  • Conventional CommitsTool

    Keep PR titles scannable for reviewers and CHANGELOG.

  • NotionTool

    Author the 2-page write-up.

Sample Ewance certificate of completion, verified by LearnCoin
Sample certificate
Your proof

Finish it. Earn this.

A verified certificate with your name. Anyone can check it.

Time to practice.

Solve this challenge
Related challenges

More like this.