001. Environment topology: QA and Prod separate VPCs

• Date: 2026-05-23
• Status: Accepted (amended 2026-07-12 — see Amendment 2026-07-12)

Decision

• In the context of deploying rag-sample to AWS as a personal demo while practicing a production-like delivery workflow, with a local Docker environment serving as development,
• facing the need for isolation between a test environment and a live one, a safe promotion path, near-zero idle cost, and minimal operational overhead for a single operator,
• we decided for two AWS environments, QA and Prod, in a single AWS account with a dedicated VPC per environment, provisioned by environment-agnostic Terraform modules (network, data, api, frontend) composed per environment with separate remote state (S3 + DynamoDB lock — the lock mechanism was later superseded, see Amendment 2026-07-12); QA deploys automatically on merge to main, and Prod is gated by a GitHub Environment manual-approval rule,
• and neglected
  • separate AWS accounts per environment via AWS Organizations — strongest isolation and the right answer for real production, but the Organizations setup, cross-account roles, and billing overhead are disproportionate for a demo,
  • a single shared VPC with tag- or subnet-based separation — cheaper and simpler, but weak blast-radius isolation between QA and Prod,
  • Prod-only (no QA) — fastest and cheapest, but no safe place to validate changes before they reach the live environment,
• to achieve
  • blast-radius isolation between QA and Prod at the network layer,
  • a realistic promotion workflow (auto QA, gated Prod) that exercises production delivery practices,
  • near-zero idle cost in each environment via scale-to-zero (Aurora Serverless v2 0-ACU auto-pause, Lambda, pay-per-token Bedrock), with QA paused harder than Prod,
• accepting
  • roughly 2x the single-environment baseline cost from running the stack twice (cost.md has the current figures),
  • a shared account boundary, so IAM and account-level limits or misconfiguration can still affect both environments, and cost attribution does not follow the VPC boundary — per-environment spend separation requires tagging and billing configuration this topology does not itself provide,
  • a manual human step before every Prod deploy.

Consequences

Terraform is structured as reusable modules (network, data, api, frontend) composed by thin per-environment roots (qa, prod), each with its own state in an S3 backend with a DynamoDB lock table (lock mechanism superseded — see Amendment 2026-07-12); a one-time bootstrap provisions that backend. Each environment owns a dedicated VPC for the Aurora cluster.

CI gains a promotion pipeline: merge to main runs tests and applies to QA; Prod apply waits on a GitHub Environment protection rule. Cost review happens before merge, on the Terraform pull request (RAG016). Per-environment spend guardrails are provided by cost-allocation tagging.

If rag-sample ever graduates beyond a demo, the migration path is to separate AWS accounts per environment under Organizations; keeping environments behind thin per-env Terraform roots makes that migration mostly a backend and provider change rather than a rewrite.

Amendment (2026-07-12): drop DynamoDB lock table for S3-native locking

S3-native locking (use_lockfile on backend "s3") reached GA in Terraform 1.11.0 (2025-02-27), before this ADR’s acceptance date. Scope is limited to backend.tf in each root and the aws_dynamodb_table.locks resource in bootstrap. The original Decision and Consequences text above still names “S3 + DynamoDB lock” as the historical decision, with an inline pointer to this amendment.

we now decide for use_lockfile = true in place of the DynamoDB lock table, accepting a one-time migration and raising required_version to >= 1.11, to achieve one fewer AWS resource per environment and no standing deprecation warning.

Migration safety: the DynamoDB and S3-lockfile mechanisms do not share a lock namespace, so a cutover mid-apply risks two un-mutually-excluded applies corrupting state. Perform the cutover only with no apply in flight — and because QA auto-applies on merge to main, freeze that pipeline during the QA cutover. Order: update backend.tf and run terraform init -reconfigure, confirm a clean plan, then remove the aws_dynamodb_table.locks resource in a subsequent apply.