QA plan: pytest coverage CI job (#15)

Story: pytest coverage + CI test job (#15)
Capability: Retrieval & generation API (#11)
ADR: RAG006 — 90% aggregate / 70% per-file, CI hard-fail

Test scope

This story is the CI mechanism that enforces RAG006; it does not add product behavior. The QA plan verifies the pipeline itself works correctly.

Type	Coverage
CI pipeline check	pytest --cov=rag_sample --cov-branch --cov-fail-under=90 --cov-report=term-missing exits non-zero when coverage drops below 90%
CI pipeline check	Per-file floor step (coverage report --fail-under=70 --skip-covered) exits non-zero when any non-excluded file is below 70%
CI pipeline check	The CI job fails a PR that introduces a file with 0% coverage
CI pipeline check	Exclusion list in pyproject.toml matches the coverage exclusion policy; __init__.py, settings.py, lambda_handler.py, export_openapi.py are omitted
Verification	Coverage report output appears in the CI job log (--cov-report=term-missing)

Entry criteria

• GitHub Actions workflow file exists for the test job
• pyproject.toml has [tool.pytest.ini_options] and [tool.coverage.run] sections

Exit criteria

• CI job runs on every PR and on merge to main
• Aggregate 90% floor: introducing a test-free module causes the CI job to fail
• Per-file 70% floor: the two-step (aggregate then per-file) both run and both gate the PR
• Exclusions: excluded modules do not appear in the coverage report denominator
• Job completes in under 5 minutes on the GitHub-hosted runner

Environments

CI only (GitHub Actions). No AWS access needed; tests use Docker Postgres service container in the workflow.

Tooling

GitHub Actions, pytest-cov, coverage. Docker Postgres service container in the workflow YAML.

Fixtures and data

None beyond what existing tests use. This story wires together what earlier stories built.

How tests are written

Verification of the CI job itself is done by:

1. A PR that intentionally drops coverage (a temporary stub module with no tests) is used once during setup to confirm the gate fires. This PR is not merged.
2. The workflow YAML is reviewed to confirm both the aggregate and per-file steps are present and blocking.

No Given/When/Then tests are written for the CI YAML itself; the verification is the observable CI run behavior.

Risks

• Docker Postgres service container startup time may cause flaky tests if not health-checked before the test step. Add options: --health-cmd pg_isready to the service definition.
• The per-file step (coverage report) depends on coverage data from the prior step; if pytest exits non-zero (below 90%), the per-file step may not run. Structure the workflow so both steps run sequentially and both contribute to the job failure status.

Capability acceptance criteria link

RAG006 is the authority. This story’s exit criteria are the direct enforcement of that ADR’s hard-fail requirement.