006. Test coverage standard: 90% line and branch, CI hard-fail

• Date: 2026-05-28
• Status: Accepted

Decision

• In the context of a Python backend (FastAPI + ingestion CLI) developed by one or more contributors with agentic assistance,
• facing the risk that generated or quickly-written code ships without meaningful test coverage, and the need for a clear, enforceable standard that applies to every contributor equally,
• we decided for a two-floor coverage standard enforced by pytest-cov as a hard-fail step in CI on every PR: a 90% aggregate floor on combined line and branch coverage over src/rag_sample/, and a 70% per-file floor on combined line and branch coverage for any non-excluded file, with module exclusions defined in a companion policy maintained outside this ADR,
  • Branch coverage is included alongside line coverage because branch misses (uncovered if/else paths) are the most common source of untested error handling
  • Hard-fail (not a warning) means a PR cannot be merged below either floor — enforcement is unconditional
  • 90% aggregate and 70% per-file are floors, not targets; coverage above the floors is fine, the floors are the minimum
  • The per-file floor prevents the aggregate from masking one or two badly-undertested files; a high-coverage module elsewhere should not “subsidize” an untested handler
  • The list of modules excluded from coverage measurement is maintained in Coverage exclusion policy and can evolve without amending this ADR; exclusions exist for code that cannot be meaningfully unit-tested (package __init__.py, settings dataclasses, Lambda entrypoints, build-time scripts)
• and neglected
  • 80% threshold — common but leaves too much room for untested error paths in a codebase where Bedrock and DB calls are the primary failure surfaces
  • 100% threshold — impractical; forces trivial tests of boilerplate (e.g. __init__.py, config dataclasses) and slows development
  • Warning-only enforcement — offers no guarantee; contributors learn to ignore it
  • Aggregate-only enforcement (no per-file floor) — was the original choice; revised because aggregate measurement allows one or two badly-undertested files to hide behind well-covered ones, defeating the floor’s purpose for the specific failure surfaces the standard exists to catch
• to achieve a baseline confidence that the non-trivial logic (chunking, repository queries, API request/response handling, provider calls) is exercised by tests before merging,
• accepting
  • Contributors must write tests before a PR can merge; generated code from agents is not exempt
  • Reaching 90% on the provider layer (Bedrock calls) requires stubbing or monkeypatching; integration tests hitting real AWS are excluded from the coverage run
  • The threshold may need revisiting if the codebase grows infrastructure-heavy modules that are difficult to unit-test

Consequences

pyproject.toml gains a [tool.pytest.ini_options] addopts entry --cov=rag_sample --cov-branch --cov-fail-under=90 --cov-report=term-missing, a [tool.coverage.run] omit list driven by the Coverage exclusion policy, and a per-file floor enforced by a small CI step that runs coverage report --fail-under=70 --skip-covered after the suite (or an equivalent check, since pytest-cov has no native per-file --fail-under). The CI test job (issue #15) runs this configuration. Developers run the same command locally. Coverage reports are printed to the terminal; no HTML artifact is required for CI to pass, but one can be generated locally with --cov-report=html.

Changes to the exclusion list do not amend this ADR. The exclusion policy page is the source of truth for what is excluded and why; this ADR is the source of truth for the 90% aggregate floor, the 70% per-file floor, and hard-fail enforcement.

Notes

The pytest-cov invocation:

pytest --cov=rag_sample --cov-branch --cov-fail-under=90 --cov-report=term-missing

pytest-cov is already in the dev dependency group (pyproject.toml).