SUBMISSION: Case 6 (Capstone) — End-to-End System Reliability: Hallucinations, LLM-as-a-Judge & Guardrails

# SUBMISSION: Case 6 (Capstone) — End-to-End System Reliability: Hallucination Diagnosis, Automated LLM-as-a-Judge Evaluation & Multi-Layer Guardrails > **Industry Context:** A live healthcare and insurance advisory assistant experienced critical escalations: hallucinating covered benefits, inventing copay amounts, and leaking system prompts under adversarial jailbreaks. Perform root-cause forensics, build an automated LLM-as-a-Judge evaluation harness, and deploy defensive guardrails. --- ## Deliverables 1. `forensic_triage.py`: Script categorizing 50 failure logs into retrieval failure, context dilution, parametric hallucination, or adversarial jailbreak. 2. `llm_judge_evaluator.py`: Automated evaluator using GPT-4 / Claude to score faithfulness and safety on a 1–5 scale with mandatory CoT rationales, counterbalanced against position bias. 3. `guardrail_pipeline.py`: Pre-execution input sanitizer (regex + prompt injection classifier) and post-execution entity/number validator verifying outputs against retrieved context. 4. `capstone_audit_report.md`: Executive reliability audit certifying production readiness. --- ## Starter Code Scaffold ```python from typing import Dict, Any def evaluate_with_judge(query: str, context: str, response: str, judge_client) -> Dict[str, Any]: """Run LLM-as-a-judge with CoT rationale preceding numeric grade (1-5) and position swap.""" prompt = f"""You are an impartial expert evaluating response faithfulness to retrieved context. Context: {context} User Query: {query} Candidate Response: {response} Analyze step-by-step whether every statement in the response is supported by the context. Then assign a Faithfulness score from 1 (completely hallucinated) to 5 (fully grounded). Return JSON: {{"reasoning": "...", "score": 1-5}} """ # TODO: Invoke judge client, parse JSON, execute counterbalanced swap pass ``` --- ## 100-Point Evaluation Rubric - **Criterion 1: Forensic Root Cause Analysis (25 pts)** — Clear taxonomy distinguishing retrieval failures, context attention limits, parametric hallucinations, and jailbreaks. - **Criterion 2: Automated LLM-as-a-Judge Evaluation (25 pts)** — CoT rationale before score, low-precision scale (1–5), position-swap counterbalancing to eliminate order bias. - **Criterion 3: Multi-Layer Guardrail Pipeline (25 pts)** — Input sanitization (injection prevention) and post-execution entity/numeric consistency verification. - **Criterion 4: Production Certification & Audit Synthesis (25 pts)** — Complete executive audit report certifying SLA compliance and deployment readiness. --- ## Implementation Guide & Case Steps > *"Customer support is escalating complaints — the chatbot is inventing order details, promising refunds it can't deliver, and citing made-up policies. Diagnose and fix."* --- ## Step 1 — Confirm the problem Pull 10-20 recent complaints. Compare what the chatbot said to ground truth: ```sql SELECT request_id, customer_id, question, bot_answer, customer_complaint FROM support_tickets WHERE escalation_reason = 'incorrect_bot_response' AND created_at >= NOW() - INTERVAL '7 days' LIMIT 20; ``` Look for patterns. Common categories: 1. **Fabricated facts**: invented order numbers, tracking links, policies. 2. **Wrong commitments**: promising refunds outside policy. 3. **Misinterpretation**: wrong assumptions about customer intent. 4. **Outdated info**: model used training-data knowledge instead of current data. ## Step 2 — Find the root cause For each category, identify the underlying issue: ### Fabricated facts Likely cause: model relies on its training rather than retrieved context. Check: - Are retrieved chunks actually relevant? - Does the prompt clearly say "use ONLY the provided context"? - Is there an "I don't know" fallback? ### Wrong commitments Likely cause: tools that should gate actions don't. Check: - Is there a check_refund_eligibility tool? Is it being called? - Does the model have direct refund authority, or does it require human approval? ### Misinterpretation Likely cause: prompt doesn't push the model to clarify. Check: - Does the prompt encourage clarifying questions? - Is there an "ambiguous - escalate" path? ### Outdated info Likely cause: not using RAG, or RAG retrieval is poor. Check: - Is the knowledge base current? - Is retrieval surfacing relevant policy docs? ## Step 3 — Diagnose with a sample Take 5 problem cases. Manually run the same query through the pipeline. Inspect: ```python def debug_request(question, customer_id): print(f"Question: {question}") print() # Retrieval chunks = hybrid_search(question) print("RETRIEVED CHUNKS:") for i, c in enumerate(chunks): print(f" [{i}] {c['source_title']}: {c['text'][:200]}...") print() # Reranking reranked = rerank(question, chunks) print("RERANKED TOP 3:") for i, c in enumerate(reranked): print(f" [{i}] {c['source_title']}: {c['text'][:200]}...") print()