QA plan: CI pipeline (#29)

Story: CI pipeline — test, Infracost diff, plan, apply per env (#29)
Capability: Environments and deployment promotion (#26)
ADRs: RAG006 — coverage gate; RAG005 — OpenAPI diff; RAG001 — env topology

Test scope

Type	Coverage
Pipeline structure	Workflow has distinct jobs: test, openapi-check, infracost, plan, apply-qa, apply-prod (or equivalent) in dependency order
Pipeline behavior	test job runs pytest with coverage gate; fails the pipeline if coverage drops below 90% aggregate or 70% per-file (RAG006)
Pipeline behavior	openapi-check job diffs committed openapi.yaml against FastAPI output; fails on drift (RAG005)
Pipeline behavior	infracost job posts a cost diff comment on the PR; the comment steps are informational and do not block merge
Pipeline behavior	The same job asserts each environment’s monthly total against the fixed-price ceiling and fails the PR when a total exceeds it (RAG016, detail in plan-infracost)
Pipeline behavior	plan job runs terraform plan on the QA workspace and posts output; tfsec/checkov run inline
Pipeline behavior	apply-qa job runs only on merge to main, not on PRs
Pipeline behavior	apply-prod job is environment-gated (requires manual approval per Story #28)
Security	pip-audit runs on every PR; high/critical findings post as PR comment (non-blocking until noise floor is established)
Security	npm audit --audit-level=high runs on every PR for the SPA workspace
Security	Secret scanner (e.g., gitleaks) runs on every PR; any detection not in the allowlist fails the build
Security	Terraform plan output posted with tfsec/checkov results; high-severity findings block merge

Entry criteria

• #20, #25 (Terraform bootstrap, IAM/secrets) complete so terraform init can succeed
• Story #15 (pytest CI job) merged so the test step has tests to run

Exit criteria

• All pipeline jobs listed above exist in the workflow YAML and run in correct dependency order
• test job failures block apply-qa (no deployment of broken code)
• Security jobs run on every PR: pip-audit, npm audit, secret scanner, tfsec/checkov
• The Infracost job’s ceiling assertion fails the PR when an environment’s monthly total exceeds the ceiling, while a failed or missing cost comment does not
• No Infracost job runs in the deploy workflow
• Prod apply is never triggered without a passing QA apply in the same run
• Pipeline completes in under 15 minutes on a standard GitHub-hosted runner (soft gate)

Environments

CI only (GitHub Actions). No local execution; the pipeline is the test subject.

Tooling

GitHub Actions, pytest-cov, coverage, openapi-diff or python -m scripts.export_openapi + diff, Infracost, terraform, tfsec, checkov, pip-audit, npm audit, gitleaks, Newman.

How tests are written

Pipeline correctness is verified by inspection and by triggering deliberate failures:

# Given a PR that intentionally drops coverage below 90%
# When the CI pipeline runs
# Then the test job exits non-zero and apply-qa is skipped

Deliberate-failure checks are performed once during pipeline setup and not merged. Ongoing, the pipeline is self-verifying: a red job on a PR with a known-bad change is evidence the gate works.

Security job verification:

# Given the CI workflow YAML for the secret-scanner job
# When parsed for the gitleaks step
grep -q "gitleaks" .github/workflows/ci.yml
# Then gitleaks is present in the workflow

Risks

• Infracost API rate limits in CI; use --no-usage-file for PRs and only generate full estimates on a schedule or on Terraform-touching PRs.
• tfsec and checkov may produce overlapping findings; decide which is the canonical failure source and treat the other as informational to avoid double-blocking noise.
• Secret scanner false positives (test fixtures, mocked tokens) must be in the allowlist before enabling as a blocking check.

Capability acceptance criteria link

RAG006 (test coverage), RAG005 (OpenAPI drift), RAG001 (env gating). Security per-feature requirements (#29/#30): all security CI steps enumerated above. This story’s exit criteria enforce all of them.