QA plan: QA environment auto-deploy (#27)
Story: QA environment — auto-deploy on merge to main (#27)
Capability: Environments and deployment promotion (#26)
ADR: RAG001 — QA auto-deploy, Prod manual gate
Test scope
Section titled “Test scope”| Type | Coverage |
|---|---|
| Pipeline behavior | Merge to main triggers the QA deploy job without manual intervention |
| Pipeline behavior | QA job runs tests before applying Terraform. No Infracost job is part of the QA deploy workflow; the cost comment and the fixed-price ceiling both run on the Terraform PR before merge |
| Pipeline behavior | QA deploy job does NOT gate on a manual approver (unlike Prod) |
| Security | QA OIDC role trust policy names repo:ArunskiOrg/rag-sample-app:environment:qa; no wildcard in sub |
| Security | Terraform state backend for QA is a separate S3 path from Prod; QA OIDC role cannot s3:PutObject against the Prod state bucket |
| Integration (manual) | A commit merged to main with no Terraform change deploys the Lambda zip and SPA bundle; CloudWatch logs confirm the new Lambda version is invoked |
| Integration (manual) | GET /healthz on the QA API base URL returns 200 after a successful deploy |
Entry criteria
Section titled “Entry criteria”- GitHub Actions workflow file for CI/CD exists (Story #29)
- QA GitHub Environment is configured in the repo settings (no required reviewer)
- OIDC provider and QA OIDC role exist in AWS (Story #25)
Exit criteria
Section titled “Exit criteria”- Merge to
maintriggers the QA job automatically (observable in Actions tab) - QA job applies Terraform, redeploys Lambda, redeploys SPA; all steps exit 0
- Security:
gh api repos/ArunskiOrg/rag-sample-app/environments/qaconfirms no required reviewer - Security: OIDC trust policy parsed in CI; no wildcard in
sub /healthzreturns 200 on QA after deploy
Environments
Section titled “Environments”CI (GitHub Actions) for the pipeline itself. QA AWS env for deployment validation.
Tooling
Section titled “Tooling”GitHub Actions, gh CLI, Terraform, AWS CLI, Newman for post-deploy smoke.
How tests are written
Section titled “How tests are written”# Given the QA GitHub Environment exists# When its protection rules are fetched via the APIgh api repos/ArunskiOrg/rag-sample-app/environments/qa \ | python3 -c "import sys,json; e=json.load(sys.stdin); reviewers=e.get('protection_rules',[]); assert not any(r['type']=='required_reviewers' for r in reviewers), 'QA env has required reviewer — should not'"# Then no required-reviewer rule is present- Aurora auto-pause in QA means the first
/healthzafter a cold start may take 20–30 s; the smoke test should retry with a backoff rather than failing immediately. - If the Terraform plan detects a resource replacement (e.g., security group), the auto-apply will destroy and recreate it without human review. Add a plan-diff step that fails the job on replacements unless the PR description includes a
[force-replace]label.
Capability acceptance criteria link
Section titled “Capability acceptance criteria link”RAG001: QA deploys automatically on merge to main. Exit criteria directly verify this, including the security requirement that QA and Prod OIDC roles are isolated and Prod state is unwritable from the QA role.
© 2026 Benjamin Arunski