Skip to content
RAG Reference Implementation

013. Safety gate via Bedrock Guardrails

Decision

Section titled “Decision”
  • In the context of enforcing a safety gate on the rag-sample query pipeline, where the LLM is a Haiku-class model via Bedrock (RAG007), the pipeline runs inside the API Lambda (RAG004), and the product needs harmful-content filtering on user input and on model output before either crosses a trust boundary,
  • facing the choice of a safety mechanism that covers input and output, that integrates with the existing Bedrock surface without adding a new vendor or a new network path, that has a coherent cost story at demo scale, and that does not silently couple the safety contract to the generation model,
  • we decided for AWS Bedrock Guardrails as the safety gate, provisioned as a single Guardrails policy per environment in Terraform, with content filters (hate, insults, sexual, violence, misconduct) and the prompt-attack filter enabled, evaluated on the question before the pipeline runs (#71) and on the generated answer before it returns (#72), via two ApplyGuardrail calls per request rather than via Guardrails inlined into the InvokeModel call,
  • and neglected
    • A Python moderation layer inside the API Lambda (regex / keyword lists, optional small classifier). Cheap at runtime, but it makes Arunski the maintainer of a safety taxonomy that has no upstream and no benchmark, drifts against threats the project will not track, and offers no defense against prompt injection beyond what its author thought to encode. Rejected on maintenance and quality grounds, not cost.
    • Amazon Comprehend toxicity / PII detection. Covers part of the surface (toxicity, PII), does not cover prompt attacks, and pulls in a second AWS service with its own pricing dimension and its own client. Weaker coverage at higher integration cost than the chosen option.
    • OpenAI Moderation API. Strong content coverage, free at the rate limits the product cares about, but adds an OpenAI vendor dependency and a network path outside the AWS trust boundary for a request that is otherwise entirely Bedrock-internal. Rejected for the same trust-boundary reason that RAG007 rejected the OpenAI generation models.
    • Guardrails inlined via InvokeModel with a guardrailIdentifier. Halves the API-call count but couples the safety contract to the generation call: the input is only guarded after it has been combined with retrieved chunks and a system prompt, and a guardrail intervention is reported as part of the model response rather than as a distinct outcome. Rejected because the project wants input-guard intervention to short-circuit the pipeline before retrieval and generation run, and wants the intervention response to be observable as its own event independent of the model invocation.
    • Denied topics in Guardrails for off-topic detection. Guardrails supports denied-topic policies, which would let the safety gate also reject questions outside the corpus’s domain. Rejected as a scope expansion: off-topic is a retrieval concern, not a safety concern, and treating it as safety would conflate “this question is harmful” with “this question has no good answer in the corpus.” Off-topic detection is handled in the retrieval layer via a score threshold and a no-match response path (#74).
    • Retrieved-chunk guarding (running ApplyGuardrail over the top-k chunks before they enter the prompt). Defensible on principle, deferred because the corpus is a vetted dataset under our control (RAG003) and the marginal safety improvement does not justify a third ApplyGuardrail call per request at this stage. Revisit if the corpus source changes or if a user-contributed corpus path is introduced.
  • to achieve
    • a managed safety policy with a real upstream (AWS) that updates filters without code changes here,
    • defense against prompt injection via the prompt-attack filter, which a hand-rolled moderation layer would not credibly provide,
    • input-side short-circuiting, so harmful questions never reach retrieval or generation and never consume their cost,
    • output-side filtering, so a model response that drifts into a filtered category is caught before it crosses the API boundary,
    • a single safety contract that survives a future change of generation model, since ApplyGuardrail is independent of InvokeModel,
  • accepting
    • two additional Bedrock API calls per request (one input, one output) instead of one, with the latency and cost that implies,
    • that the safety policy is opaque relative to a hand-rolled layer: filter thresholds are AWS-defined, and policy drift between AWS releases is something we inherit rather than control,
    • that retrieved chunks are not guarded today, with the limited risk that implies given a vetted corpus,
    • that off-topic detection is not a Guardrails concern and must be implemented and tested separately in the retrieval layer.

Cost

Section titled “Cost”

Guardrails pricing is per 1,000 text units evaluated, with separate line items for content filters and the prompt-attack filter. With two ApplyGuardrail calls per request (input on the question, output on the answer) and demo-scale volumes (~1k–10k queries / month / environment), Guardrails is the smallest line item in the per-environment Bedrock bill.

A Dollar / environment / month ceiling applies to Guardrails spend (in the range of tens of dollars), tracked alongside the rest of the per-environment cost envelope in #64. Crossing the ceiling triggers a review of request volume and of whether retrieved-chunk guarding has been quietly enabled.

Consequences

Section titled “Consequences”

The API Lambda (RAG004) gains two ApplyGuardrail calls in its query path:

  1. Input guard (#71) — called on the raw user question after authorization (RAG009) and before retrieval. On intervention, the pipeline short-circuits: no retrieval, no generation, no Aurora reads beyond what authorization already performed.
  2. Output guard (#72) — called on the model-generated answer after InvokeModel returns and before the response is serialized. On intervention, the answer is discarded and the same intervention response shape is returned.

Either intervention returns HTTP 400 with a body shaped per #73:

{
  "error": "content_policy_violation",
  "stage": "input" | "output",
  "message": "<safe, non-leaky description>"
}

The stage field distinguishes input-guard from output-guard interventions so the frontend can surface an appropriate message without inferring stage from timing. The message is a fixed, non-leaky string per stage; it does not echo the offending content and does not enumerate which filter triggered.

This response shape is part of the API contract and is documented in the OpenAPI spec (RAG005) as a 400 response with a ContentPolicyViolation schema, shared by both guard stages.

The Guardrails policy itself is Terraform-managed (#70) per environment. Policy changes ship through the same Terraform pipeline as the rest of the infrastructure; the API Lambda reads the guardrail identifier and version from pydantic-settings. The IAM role for the API Lambda gains bedrock:ApplyGuardrail on the guardrail ARN, scoped per environment.

Observability (RAG008) records each intervention as a structured Powertools log event with stage, guardrail action, and matched filter category; X-Ray segments cover the ApplyGuardrail calls so input-guard latency is visible independently of generation latency.

Relationship to off-topic detection

Section titled “Relationship to off-topic detection”

Off-topic detection is not a Guardrails feature in this product. A question that is on-policy but unanswerable from the corpus (no chunk clears the retrieval score threshold) returns a distinct no-match response from the retrieval layer (#74), not a content_policy_violation. The two outcomes are observable as separate events and tested separately. If the product ever needs to harden the boundary of what topics it will engage with at all (regardless of whether the corpus could answer), denied topics in Guardrails becomes the right tool and gets its own ADR.

Open questions

Section titled “Open questions”
  • Whether to enable retrieved-chunk guarding once a user-contributed or third-party corpus path is introduced. Deferred until such a corpus exists.
  • Whether to differentiate the user-facing message per filter category (e.g., distinct copy for prompt-attack vs content-filter interventions) once the frontend has live UX feedback. Deferred until the frontend ships.

© 2026 Benjamin Arunski