QA plan: /query endpoint (#13)

Story: /query endpoint — retrieve + Claude synthesis with citations (#13)
Capability: Retrieval & generation API (#11)
Spec: specs/retrieval.md
Security ADR: RAG009 — bearer-token auth on this route

Test scope

Type	Coverage
Unit	Happy path: FakeEmbeddingProvider + FakeLLMProvider + pre-seeded DB returns {"answer": "...", "sources": [...]} with at least one source
Unit	HTTP 422 on empty question
Unit	HTTP 422 on question exceeding max length
Unit	HTTP 503 with clear message when LLMProvider.complete raises a Bedrock credentials error
Unit	Sources include content, source, metadata, similarity fields
Unit	System prompt instructs Claude to answer only from provided chunks and cite by index — verified by asserting the prompt string passed to the stub contains required instructions
Unit	retrieval_top_k setting is respected: stub DB returns exactly top_k rows
Integration	Against Docker Postgres with pre-embedded seed: /query returns non-empty sources list
Integration	Auth: requests without Authorization: Bearer header receive 401 (authorizer behavior, tested via the authorizer Story #60; referenced here as a dependency)
Contract	/query request and response schemas match committed openapi.yaml (RAG005); bearer security scheme present on the route
Security (B3)	Prompt construction test: retrieved chunk content appears inside delimited fences in the assembled prompt; user question appears after, not inside, the fence
Security (B3)	Log-scrub: test that Authorization header is absent from captured Powertools log records (RAG009 + RAG008; covered by Story #62 but integration-verified here)

Entry criteria

• #10, #12 merged and passing
• Pre-embedded seed data in tests/fixtures/embeddings_seed.sql
• Authorizer Story #60 tests passing (for auth integration check)

Exit criteria

• All unit tests pass; api/routes/query.py exceeds 70% per-file floor
• Integration: /query with seeded DB returns HTTP 200 with answer and at least one sources entry
• HTTP 422 on empty and oversized questions (confirm max length bound is meaningful, e.g. 2KB per security assessment gap)
• HTTP 503 on missing Bedrock credentials
• Prompt-construction security test passes
• OpenAPI diff check passes (spec includes bearer security scheme on POST /query)
• Aggregate coverage over rag_sample/api/ at or above 90%

Environments

Unit and integration: local with Docker Postgres. Contract: CI. No live Bedrock in CI; Bedrock stubbed at boto3 call site. Live Bedrock smoke test run manually against QA env before Prod gate.

Tooling

pytest, FastAPI TestClient, pytest-cov. Newman for contract validation of the spec file.

Fixtures and data

• postgres_db fixture with embeddings seed loaded
• FakeEmbeddingProvider: returns [[0.0]*1024]
• FakeLLMProvider: returns {"answer": "Test answer.", "sources_used": [0]}
• make_test_key() helper for auth-integrated tests

How tests are written

def test_query_returns_sources(client_with_seed_db, test_bearer_token):
    # Given the DB has pre-embedded chunks and a valid key with remaining_requests=5
    # When POST /query is called with a valid question and bearer token
    response = client_with_seed_db.post(
        "/query",
        json={"question": "What is retrieval augmented generation?"},
        headers={"Authorization": f"Bearer {test_bearer_token}"},
    )
    # Then response is 200 with answer and at least one source
    assert response.status_code == 200
    body = response.json()
    assert body["answer"]
    assert len(body["sources"]) >= 1
    assert all(k in body["sources"][0] for k in ["content", "source", "metadata", "similarity"])

Risks

• Prompt injection (T1, T2): delimiter-based mitigation must be tested explicitly; easy to verify it exists in the assembled string but subtle if the delimiter is inconsistently applied.
• Vector similarity correctness depends on the seed embeddings being at known distances. Use seed vectors with known cosine similarity ordering so the top_k ordering assertion is deterministic.
• Auth dependency on Story #60: if the authorizer is not integrated via the Lambda event in TestClient, the 401 path may not be exercisable at the unit level. Plan for a separate integration smoke test against QA env.

Capability acceptance criteria link

Spec: POST /query returns {"answer": "...", "sources": [...]} with at least one source; 422 on empty/oversized; 503 on no Bedrock credentials. All are exit criteria here. Auth 401 behavior per RAG009 is tested in Story #60 and referenced as a dependency.