5.1.2. Hallucination Reduction and Output Validation

💡 First Principle: Hallucination is a system design problem as much as a model quality problem. The FM's confidence in its output does not correlate with factual accuracy — it generates text that is statistically consistent with its training distribution, whether or not that text is factually correct. System design must compensate for this.

Hallucination reduction architecture — layered approach:

Automated Reasoning Checks — the highest-confidence grounding approach: For use cases requiring logical consistency (not just factual grounding), Bedrock's Automated Reasoning check formally verifies that FM outputs don't contradict a defined set of rules or policies:

# Bedrock Guardrails with automated reasoning for policy compliance
guardrail_config = {
    "automatedReasoningPolicyConfig": {
        "type": "ENABLED",
        # Define your policy rules that responses must comply with
        "policyArn": "arn:aws:bedrock:us-east-1:123456789:policy/insurance-policy-rules"
    }
}

Confidence scoring and semantic similarity validation:

def validate_response_grounding(response_text, retrieved_chunks, threshold=0.7):
    """Verify FM response is semantically grounded in retrieved context."""
    response_embedding = embed_text(response_text)
    
    max_similarity = 0
    best_chunk = None
    for chunk in retrieved_chunks:
        chunk_embedding = embed_text(chunk)
        similarity = cosine_similarity(response_embedding, chunk_embedding)
        if similarity > max_similarity:
            max_similarity = similarity
            best_chunk = chunk
    
    if max_similarity < threshold:
        # Response not grounded in retrieved context — potential hallucination
        cloudwatch.put_metric_data(
            Namespace='GenAI/Quality',
            MetricData=[{'MetricName': 'LowGroundingScore', 'Value': 1, 'Unit': 'Count'}]
        )
        return False, max_similarity
    
    return True, max_similarity

⚠️ Exam Trap: Bedrock Guardrails' grounding check evaluates whether the response is supported by the retrieved context that you provide in the guardrail request — but it cannot check whether the retrieved context itself is accurate. If your RAG pipeline retrieves wrong or outdated documents, the grounding check passes (response is supported by retrieved context) while the answer is still factually wrong. Grounding validation and retrieval quality validation are separate concerns.

Using the grounding check: grounding is an inference-time control. It constrains and checks each response against context you supply and changes no model weights, unlike fine-tuning. The contextual grounding check compares three inputs: the grounding source, the user query, and the content to guard (the model response). With Converse you mark the source and query with the grounding_source and query qualifiers. With ApplyGuardrail, which calls no model, you also pass the response as an unqualified or guard_content block. A guardrail has no knowledge-base setting and never retrieves context itself, so pass the retrieved chunks as the grounding source; several grounding-source blocks are evaluated together. The threshold sets how strict the check is. Use a high threshold for strict factual Q&A, where an unsupported claim is costly. Use a lower one where summarization or synthesis must paraphrase and infer beyond the literal source.

Reflection Question: Your legal document review system produces accurate responses 95% of the time when it uses retrieved context, but 3% of responses contain fabricated case citations that look real. Guardrails grounding is already enabled. What additional validation layer would you add to catch fabricated citations specifically?