A team I worked with last year hit 92% test coverage and threw a small celebration. Three weeks later, a rounding bug in their billing service quietly overcharged about 4,000 customers for six days before a support ticket surfaced it. Every line that shipped that bug was "covered." The tests confirmed the code ran — not that it was correct.
Coverage is the metric everyone can compute and almost nobody should worship. It tells you which lines executed during a test run. It says nothing about whether your assertions mean anything, whether a test would actually catch a regression, or whether what you built is what the user needed. Manage quality by a single coverage number and you'll optimize the gauge instead of the engine.
Why the coverage number lies
Coverage answers one narrow question: did this line execute? A test can call a function, assert nothing that matters, and still turn the line green. I've reviewed suites sitting at 85% where half the tests had no meaningful assertions — they were checking that the code didn't throw, which is a very low bar.
Then there's the target trap. The moment 80% coverage becomes a merge gate, engineers write tests to hit 80%, not to catch bugs. You get tests for trivial getters and untested edge cases in the gnarly payment logic, because the easy lines are cheaper to cover. The number goes up; the risk doesn't come down.
Coverage tells you what you tested, never how well you tested it. High coverage with weak assertions is just confidence you haven't earned.
Metrics that actually predict quality
The metrics worth tracking share a trait: they measure outcomes and behavior, not activity. A few I lean on in my engagements:
- Defect escape rate — the share of bugs found in production versus caught before release. This is the closest thing QA has to a north star. Trending down means your safety net is genuinely catching things.
- Mutation score — mutation testing deliberately breaks your code (flips a
<to a>, negates a boolean) and checks whether any test fails. If your tests survive the sabotage, they weren't testing much. It's the honest answer to "are my assertions real?" - Change failure rate and MTTR — straight from the DORA research: what fraction of deploys cause an incident, and how fast you recover. These tie quality to delivery, which is where it belongs.
- Flaky test rate — the percentage of tests that pass and fail without any code change. Flaky tests train your team to ignore red builds, and that quietly dismantles every other quality control you have.
Notice that none of these are free to game. You can't fake a low escape rate by writing hollow tests.
Leading indicators beat lagging ones
Escape rate is honest but slow — you learn about the gap after your users do. The real leverage is instrumenting earlier. Mutation score and assertion density are leading indicators: they tell you today whether the tests you're writing will hold up. Pair them with review-stage signals — how many defects code review catches, how long a PR sits open, how big the average change is. Smaller, faster changes fail less often. That's not folklore; it's one of the most durable findings in the DORA data.
The point isn't to track everything. Pick two or three metrics that map to your actual failure mode. A team drowning in production incidents should watch escape rate and MTTR. A team sitting on a bloated, trusted-but-untested suite should run mutation testing and find out how much of that green is real.
Where to start Monday
Don't rip out your coverage tooling — just demote it from goal to guardrail. Then add one signal that measures whether your tests actually work. My usual first move: run a mutation testing tool on your most critical module — billing, auth, whatever would make the news if it broke — and look at the survivors. The mutants that live through your suite are a precise map of where your tests are theater.
Pick the one metric that would have caught your last painful production bug, and start measuring it this week.