009. API authentication: bearer-token Lambda authorizer + Aurora-backed key store

Date: 2026-06-01
Status: Accepted
Builds on: RAG001. Environment topology, RAG002. Backend application stack and tooling, RAG004. Lambda ASGI adapter - Mangum, RAG005. OpenAPI spec governance, RAG008. Observability

Decision

In the context of a publicly reachable RAG demo whose generation costs are usage-driven (RAG007 Haiku via Bedrock; RAG008 cost shape), where the operator wants to control who hits the endpoint, cap per-user lifetime spend, and stand up a Prod-credible auth model without retreating from the HTTP API choice in RAG004,
facing the choice of authentication primitive on /query, with the operator’s requirement being operator-issued bearer tokens, each with a specified lifetime quota in the tens-to-hundreds range and a specified expiry date (whichever comes first),
we decided for
- HTTP API stays as the gateway (no switch to REST API),
- a Lambda authorizer validates a bearer token on every /query request,
- keys live in an API Keys table that includes quota and expiry,
- the authorizer decrements remaining requests atomically inside the validation transaction; allow only when the key is neither expired (date) nor exhausted (count) AND the secret hash matches,
- authorizer response caching is disabled in API Gateway because the decrement-on-call model is incompatible with caching,
- operator-side CLI generates the key, prints it once, and inserts the row,
- the SPA stores the key in localStorage and keeps it visibly displayed at all times (not behind a settings panel) so the user is continuously reminded to rotate before it expires or burns out,
and neglected
- API Gateway REST API + Usage Plans + API keys — looks like the right primitive in name, but the quota mechanic is per-timeperiod not lifetime-based, and API keys have no TTL; neither half of the operator’s “quota + expiry” model is native to Usage Plans, so the feature does not actually do what is needed,
- Cognito User Pools — full user-pool semantics (sign-up, recovery, federation) are overkill for a demo using ten-request operator-issued keys,
- JWT authorizer on HTTP API — would need a JWT issuer (Cognito or self-issued); fits a longer-lived session model rather than a burnable-quota key,
- DynamoDB for the key store — a fine fit (atomic counter, zero-ACU pricing) but adds a second data tier; Aurora is already in the system and has the access pattern (Data API),
- Authorizer caching enabled — would amortize the Aurora lookup but defeats the decrement-on-call semantics; any cache hit past the quota silently allows free requests,
- No authentication (the prior posture) — simpler but leaves the Bedrock bill exposed to any caller; the per-key quota is the primary cost defense in this model, not the rate limit or the spend alarm,
to achieve
- per-user lifetime spend cap: request quota. For example, 10 requests × ~$0.0035 per Haiku-class query = ~$0.035 maximum exposure per issued key, regardless of misuse,
- operator control over who reaches the endpoint without standing up a user-pool product,
- a Prod-credible auth model that does not contradict RAG001 or RAG004 (HTTP API + Mangum stay),
- a visible-key UX that turns the expiry date into a user-facing reminder rather than a silent failure mode,
accepting
- authorizer adds ~20–50 ms to every request for the cold-path Aurora Data API lookup (acceptable against the ~1–3 s Haiku generation latency from RAG007),
- key compromise = limited exposure (~$0.035 plus 10 requests of usage at a 10-request quota); tolerable because the operator personally hands keys to identified recipients and can revoke a row at any time,
- operator workflow has a manual hand-off step (the CLI prints the key; the operator emails it); not automated by design,
- SPA carries the bearer token in localStorage and re-attaches on every request — standard browser-storage exposure (XSS would leak the key); mitigated by RAG-sample’s small attack surface and the per-key spend cap,
- authorizer cache is off, so every request pays the lookup; with Aurora Serverless v2 auto-pause this can mean cold-resume cost when the cluster has been idle (rare at expected demo cadence).

Consequences

A new authorizer Lambda function joins the deployment: distinct from the API Lambda, with its own IAM role scoped to rds-data:ExecuteStatement on the cluster ARN + secretsmanager:GetSecretValue on the DB credential secret. The Terraform api/ module gains the authorizer function, its IAM role, the HTTP API authorizer resource pointing at it, and a route-level binding on POST /query (authorizer absent from /healthz).

New api_keys table. The operator-side CLI is added to the app’s tools package; it shares config and DB access with the API and ingestion runtimes per RAG002.

The C4 model gains an authorizer application in ragSample.

RAG005 OpenAPI governance is triggered: the FastAPI app gains a bearer security scheme applied to /query; the committed openapi.yaml regenerates accordingly, and the CI diff check holds.

RAG008 gains a follow-up control: the Authorization header MUST be scrubbed from CloudWatch logs at the Logger configuration layer. A failing test on log output is part of the authorizer Story’s acceptance.

The trust boundaries “Browser → API Gateway” row changes posture from “no authn” to “authenticated bearer-token; per-request authorizer lookup”. The threat model gains two threats (key leakage in logs; shared-key spend exposure).

The Web UI spec gains the always-visible-key element. The retrieval spec gains the 401 response shape on missing/invalid/expired/burned-out keys.

Notes

Cost shape addition (per environment)

Item	Cost	Notes
Authorizer Lambda invocation	~$0 at demo volumes	Within free tier; one extra invocation per `/query`
Authorizer ↔ Aurora Data API call	included in Aurora line of cost model	One extra read + one update per request
API Gateway authorizer feature	included in HTTP API request charge	No additional GW fee

Net: rounding error on top of RAG007 + RAG008. Update to cost.md is a one-line note under the API Gateway row.

Re-evaluation triggers

The request quota becomes a friction point (operator hands out keys more often than reissuance can keep up) → consider raising the quota or moving to a daily-quota model on a Usage Plan with REST API.
The operator wants to delegate key issuance (the manual CLI step becomes a bottleneck) → revisit with a small self-service Lambda + Cognito or magic-link flow.
Streaming responses become a requirement (RAG004 revisit) — the bearer-token model continues to apply; the authorizer architecture is unaffected.
XSS in the SPA becomes a real concern (e.g., richer rendering, third-party libraries) — revisit localStorage for the key in favor of an HTTP-only cookie pattern, which requires a backend session layer.

Amendment log

2026-08-06 — Corrected the per-query cost used in the spend-cap arithmetic from ~$0.001 to ~$0.0035, and the derived per-key exposure from ~$0.01 to ~$0.035. The original figure came from the Claude 3 Haiku rate; the deployed model is Haiku-class at $1/M input · $5/M output (RAG007).