Agentic Design

Patterns
🔧

Corrective RAG (CRAG)(CRAG)

RAG system that automatically detects and corrects poor retrieval results through quality assessment and re-retrieval

Complexity: highKnowledge Retrieval (RAG)

🎯 30-Second Overview

Pattern: Enhanced RAG with explicit retrieval quality evaluation and corrective actions based on confidence scoring

Why: Reduces hallucinations and improves accuracy through quality assessment and adaptive correction strategies

Key Insight: Three-tier correction strategy: refine high-confidence, supplement medium-confidence, re-retrieve low-confidence results

⚡ Quick Implementation

1Initial Retrieval:Retrieve candidate documents using dense/sparse/hybrid search
2Quality Assessment:Evaluate retrieval quality with confidence scoring (high/medium/low)
3Corrective Action:Refine, supplement, or re-retrieve based on confidence band
4Knowledge Refinement:Decompose and recompose evidence for optimal context
5Generate & Verify:Generate response with citations and optional verification
Example: query → retrieve → evaluate_quality → [correct/supplement/re-retrieve] → refine → generate

📋 Do's & Don'ts

✅Implement explicit retrieval quality evaluator with calibrated confidence thresholds
✅Use knowledge refinement with decompose-then-recompose for evidence processing
✅Apply web search supplementation for medium confidence retrieval results
✅Enforce strict citation requirements with provenance tracking
✅Cache evaluator outputs and refined knowledge for efficiency
❌Allow evaluator miscalibration without regular confidence score validation
❌Skip query drift prevention during corrective re-retrieval
❌Create unbounded correction loops without cost and latency controls
❌Mix outdated and fresh sources without temporal reconciliation
❌Neglect abstention mechanisms when confidence remains persistently low

🚦 When to Use

Use When

  • • High-stakes applications requiring verified accuracy and provenance
  • • Rapidly changing domains with frequent content updates
  • • Noisy or heterogeneous knowledge bases with quality variation
  • • Long-tail queries where initial retrieval often fails
  • • Regulated environments requiring explicit evidence grounding

Avoid When

  • • Real-time applications with strict latency requirements
  • • Closed-book tasks where parametric knowledge suffices
  • • High-quality homogeneous corpora with consistent recall
  • • Environments prohibiting external web access for supplementation
  • • Simple factual queries with reliable standard RAG performance

📊 Key Metrics

Answer Faithfulness
Factual correctness and groundedness in retrieved evidence
Evaluator Calibration
Accuracy of quality confidence predictions (ROC-AUC, ECE)
Correction Effectiveness
Quality improvement from corrective actions
Retrieval Precision
Relevance of documents after correction
Citation Coverage
Percentage of claims supported by evidence
Action Distribution
Balance of use/supplement/re-retrieve decisions

💡 Top Use Cases

Legal Research: Case law analysis requiring verified sources and temporal accuracy
Medical Q&A: Clinical decision support with evidence-based recommendations and safety checks
Financial Analysis: Market research combining real-time data with historical knowledge
Policy Research: Government and regulatory information requiring up-to-date accuracy
Technical Documentation: Software and API documentation with version-specific corrections

References & Further Reading

Deepen your understanding with these curated resources

Foundational Papers & CRAG Research

Corrective Retrieval Augmented Generation (Yan et al., 2024)

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (Asai et al., 2023)

Active Retrieval Augmented Generation (Jiang et al., 2023)

Quality Assessment & Evaluation

Searching for Best Practices in Retrieval-Augmented Generation (Barnett et al., 2024)

RAGAS: Automated Evaluation of Retrieval Augmented Generation (Es et al., 2023)

TruLens: Evaluation and Tracking for LLM Applications

RGB: A Comprehensive Evaluation Benchmark for RAG Systems (Chen et al., 2024)

Knowledge Refinement & Processing

Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023)

LongLLMLingua: Accelerating Large Language Model Inference via Prompt Compression (Jiang et al., 2023)

Chain-of-Verification Reduces Hallucination in Large Language Models (Dhuliawala et al., 2023)

Factuality Enhanced Language Models for Open-Ended Text Generation (Lee et al., 2022)

Implementation Frameworks & Tools

LlamaIndex Corrective RAG Workflow Implementation

LangChain RAG Evaluation and Correction Chains

Haystack Document Evaluation and Reranking Components

DSPy: Optimizing LM Prompts and Weights for RAG Systems

Web Search Integration & Supplementation

Bing Search API for Real-Time Information Retrieval

Google Search API Integration for Knowledge Supplementation

Tavily Search API for AI Applications

You.com Search API for Developer Integration

Production Deployment & Monitoring

LangSmith: Production Monitoring for RAG Applications

Weights & Biases: Experiment Tracking for CRAG Systems

Arize AI: ML Observability for RAG Performance Monitoring

Neptune: Experiment Management for RAG System Optimization

Contribute to this collection

Know a great resource? Submit a pull request to add it.

Contribute

Patterns

closed

Loading...

Built by Kortexya