Leaving Codecov

Agents write a lot of the code in my repos now. The PRs come in with passing tests and a green coverage badge, and for a while I merged them without thinking much about it. Then I kept finding merged code that no test had ever run, usually while I was debugging something else.

The badge wasn't lying. On a repo of any size, ten new untested lines don't move project coverage: 88% before the merge, 88% after. The number that actually moves is patch coverage, how many of the new executable lines in this PR ran. Nothing in my setup gated on it. An agent can add a hundred lines, cover twenty of them, and every check stays green.

Why I built tested.dev

I built tested.dev because I wanted two things. First, to close those gaps with the agents themselves: an agent that can see which of its new lines never ran will write the missing test in the same session, while the context is still loaded. Second, visibility on the problems a badge never surfaces, like flaky tests and a suite that quietly gets slower. Coverage is the only thing that gates a PR. The Tests and Performance tabs are there to look at; neither one fails anything.

The demo PR

The demo PR is that story with real numbers. Project coverage is 88.1% against an 85% floor, so it passes. Patch coverage is 42.7% against an 80% floor, and it fails: session.ts lines 12-21 never ran. The Tests tab shows two flaky tests, the Performance tab shows where the suite time goes, and neither blocks the merge. Coverage is the only check that fails.

Coming off Codecov

Codecov was never the problem here. I ran it for years and the reports were accurate. But once the gate moved into the agent's loop and onto the PR, running both made no sense, so my repos came off. The part I expected to miss was the dashboard; it turned out app.tested.dev/repos keeps a 30-day mainline chart and the PR list for each repo, so I didn't. The migration is mostly one workflow edit, and the exact steps are in the how-to.

The next post, Patch coverage for coding agents, is the loop itself: what an agent does when tested check fails in its session. After that, the GitHub Action, which is the same check as a required job on every PR, whether or not anyone ran it locally.