QA plan: CloudWatch logging and basic metrics (#34)

Story: CloudWatch logging and basic metrics (#34)
Capability: Cost guardrails and observability (#31)
ADR: RAG008 — AWS-native CloudWatch + X-Ray + Lambda Powertools

Test scope

Type	Coverage
Unit	Lambda Powertools Logger is used; no print() calls exist in any Lambda handler
Unit	Log scrub: a request fixture carrying a fake Authorization: Bearer test-token header produces a log entry with no authorization key under any casing (per RAG009 follow-up and security per-feature requirements)
IaC (plan review)	Every CloudWatch log group has retention_in_days = 30 set explicitly; no default-infinite retention
IaC lint	tfsec rule: retention_in_days must be present and non-zero on all aws_cloudwatch_log_group resources
Integration (manual, QA)	After a POST /query call: CloudWatch Logs Insights query returns a structured log entry with request_id, level, and message fields; no authorization field
Integration (manual, QA)	CloudWatch alarm on Lambda error rate exists and is in OK state with no errors present
Integration (manual, QA)	The error-rate alarm SNS topic has at least one confirmed subscriber

Entry criteria

• #12, #13 (FastAPI app) and #23 (API module Lambda) deployed to QA
• Story #25 (IAM) complete so Powertools can write to CloudWatch

Exit criteria

• Unit tests: log-scrub test passes; no print() in Lambda handlers (verified by ruff or grep)
• All log groups in Terraform have retention_in_days = 30 (tfsec passes)
• Integration: CloudWatch Logs Insights query confirms structured log shape and absence of authorization field
• Lambda error-rate alarm exists in QA and is in OK state at verification time
• SNS subscription confirmed

Environments

Unit: CI (Docker Postgres not needed for log tests). IaC lint: CI. Integration: manual against QA.

Tooling

pytest, pytest-cov, tfsec, AWS CLI (aws logs start-query, aws cloudwatch describe-alarms), Lambda Powertools.

How tests are written

def test_authorization_header_scrubbed(lambda_log_capture):
    # Given a Lambda event containing an Authorization header with a fake token
    event = {"headers": {"Authorization": "Bearer rks_test_fake"}, "body": '{"question": "test"}'}
    # When the handler processes the event (using a stubbed provider)
    handler(event, {})
    # Then no log entry contains the authorization key under any casing
    for record in lambda_log_capture.records:
        record_str = json.dumps(record).lower()
        assert "authorization" not in record_str, f"Authorization header found in log: {record}"

Risks

• Lambda Powertools version drift: a new Powertools major version may change the log format; pin the version in pyproject.toml and test the log shape explicitly.
• CloudWatch Logs Insights queries have a latency of up to 5 minutes; the manual verification step should wait for log ingestion before running the query.

Capability acceptance criteria link

RAG008: structured logs via Lambda Powertools, X-Ray traces, CloudWatch alarms. RAG009: Authorization header scrubbed from logs. Security per-feature requirements (#31–#34): log-scrub test, 30-day retention, alarm with subscriber. All exit criteria map to these requirements.