Evaluating Hallucination and Citation Faithfulness in Retrieval-Augmented Generation

1. The Evaluation Crisis in Production RAG

A standard proof-of-concept RAG pipeline can be assembled in an afternoon using off-the-shelf libraries. However, moving from a demo to a production system capable of handling thousands of varied user inquiries exposes reliability problems:

• Silent Hallucinations: The model produces an articulate, convincing answer that includes plausible facts not present in the retrieved reference context.
• Citation Hallucination: The model outputs superscript citation tags (e.g., [1], [2]) pointing to source documents that do not actually contain or support the asserted claim.
• Context Distraction: When retrieved chunks contain irrelevant information or conflicting viewpoints across documents, generative models often select the incorrect perspective or blend contradictory numbers.

Without continuous, automated evaluation metrics, engineering teams cannot safely refactor chunking strategies, swap embedding models, or update prompt templates without risking quality regressions.

2. The Four Core RAG Evaluation Metrics

RAG evaluation is commonly organised around four metrics (they overlap and are not independent):

       User Query ─────────────────► [Retrieval Engine]
           │                                 │
           │                                 ▼
           │                         Retrieved Contexts
           │                       (Context Relevance)
           ▼                                 │
   [LLM Generator] ◄─────────────────────────┘
           │
           ▼
    Generated Answer
   (Faithfulness & Answer Relevance)

1. Context Relevance: Measures the proportion of retrieved chunks that are genuinely relevant to answering the user query:

2. Faithfulness (Grounding): Measures whether every statement in the generated answer is supported by the retrieved context:

3. Answer Relevance: Measures whether the generated output directly resolves the user's initial question without deviating into tangential topics.
4. Citation Recall & Precision:

• Citation Recall: Proportion of statements in the answer that require attribution and possess an associated citation tag.
• Citation Precision: Proportion of provided citation tags that actually support the specific claim they are attached to.

3. Citation Grounding & Faithfulness Mathematics

To compute Faithfulness algorithmically:

1. Decompose the generated response A into a set of atomic propositional claims: C = {c₁, c₂, ..., cₘ}.
2. For each claim cᵢ, evaluate its entailment against the retrieved context passage set P = {p₁, p₂, ..., pₖ}:

If Faithfulness Score < 1.0, the pipeline flags the output for automated revision, citation re-verification, or execution refusal.

4. Building an Automated Offline Evaluation Harness

Below is a minimal sketch that checks citation structure only: whether each sentence carries a citation tag, and whether the tags point to chunks that were actually retrieved. It does not check whether a cited chunk supports the claim. That needs an entailment model, quote matching or human review (the Agentic RAG project, for example, checks citations against verbatim quotes).

from typing import List, Dict, Set, Tuple
import re

class CitationStructureChecker:
    def extract_citations(self, answer_text: str) -> List[Tuple[str, List[str]]]:
        """
        Parses sentences and extracts associated bracketed source chunk IDs.
        Example sentence: 'DuckDB uses columnar storage [chunk_01][chunk_03].'
        """
        sentences = [s.strip() for s in re.split(r'(?<=[.!?]) +', answer_text) if s.strip()]
        extracted = []
        
        for sent in sentences:
            chunk_ids = re.findall(r'\[(chunk_\d+)\]', sent)
            clean_sentence = re.sub(r'\[chunk_\d+\]', '', sent).strip()
            extracted.append((clean_sentence, chunk_ids))
            
        return extracted

    def evaluate_citation_structure(
        self, 
        answer_text: str, 
        retrieved_chunks: Dict[str, str]
    ) -> Dict[str, float]:
        """
        Computes citation coverage and the share of citation tags that point to retrieved chunks.
        """
        extracted = self.extract_citations(answer_text)
        if not extracted:
            return {"citation_coverage": 0.0, "valid_citation_share": 0.0}

        valid_chunk_ids: Set[str] = set(retrieved_chunks.keys())
        total_claims = len(extracted)
        claims_with_citations = 0
        valid_citations = 0
        total_citations_provided = 0

        for sentence, cited_ids in extracted:
            if cited_ids:
                claims_with_citations += 1
                for cid in cited_ids:
                    total_citations_provided += 1
                    if cid in valid_chunk_ids:
                        valid_citations += 1

        citation_coverage = claims_with_citations / total_claims if total_claims > 0 else 0.0
        valid_citation_share = (
            valid_citations / total_citations_provided 
            if total_citations_provided > 0 else 0.0
        )

        return {
            "citation_coverage": round(citation_coverage, 3),
            "valid_citation_share": round(valid_citation_share, 3),
            "sentences_evaluated": total_claims
        }

5. Testing with Budget Limits & Mocked APIs

Running evaluation suites against a live commercial LLM API on every commit costs money and can hit rate limits.

The Mocked Embedding & Test Double Strategy:

1. Deterministic Mock Embeddings: For retrieval test suites, replace live vector APIs with deterministic hashing encoders (e.g., token-count-based orthogonal vectors or pre-computed embeddings cached in SQLite).
2. Offline Regression Tests: Cover query planning, hybrid RRF fusion and citation schema validation with tests that make no network calls. In the Agentic RAG project, 87 of its 90 test functions run offline by default; the 3 live integration tests are opt-in.
3. Hard Budget Limits: Make live runs opt-in and cap their spend. The Agentic RAG project's benchmark runner needs an explicit --real-api flag and enforces a spend cap (default $0.50).

6. Handling Contradictions & Insufficient Evidence

A resilient RAG system must handle the case where evidence is insufficient or contradictory.

The Refusal Protocol:

When the retrieved evidence is judged insufficient (for example by an auditing step, or by a retrieval-score threshold tuned on your own data), the pipeline should return an explicit Refusal State:

"The retrieved documentation does not contain sufficient evidence to answer this question accurately. Please refine your query or provide the relevant source documents."

Refusing when evidence is weak is usually better than producing an unsupported answer, but it has a cost. In the Agentic RAG project's held-out test set (32 questions, frozen results, small sample), the full system refused all 4 unanswerable questions, where the dense and hybrid baselines refused none. It also wrongly refused 3 of 24 answerable questions and scored 45.8% answer accuracy, against 58.3% and 60.4% for the baselines.

7. Continuous Evaluation Pipeline Architecture

In a modern CI/CD pipeline:

1. Pull Request Stage: Run the offline test suite (query decomposition, schema validation and citation matching with mocked vector indices) on every change.
2. Scheduled Evaluation Stage: Run a fixed, versioned question set against the live pipeline on a schedule, and record faithfulness, retrieval recall and latency percentiles so regressions show up over time. Keep a held-out split that is not used for tuning, as the Agentic RAG project does (8 development and 32 held-out test questions).