Chain-of-Verification

Chain-of-Verification (CoVe) is a method proposed to reduce the number of hallucinations (the generation of factually incorrect but plausible responses) in large language models (LLMs). The approach, developed by a team of researchers from Meta AI led by Shehzaad Dhuliawala and presented in the 2023 paper "Chain-of-Verification Reduces Hallucination in Large Language Models," belongs to the class of self-verification and self-correction methods for LLMs. The core idea of CoVe is for the model to perform a step-by-step verification of its generated response without relying on external sources. This encourages the system to expend more "reasoning" effort on self-analyzing its response and correcting its own errors before presenting the answer to the user.

Background: Hallucinations in Language Models

Large language models (LLMs) often suffer from the phenomenon of "hallucinations"—generating responses that appear plausible but are factually incorrect. This problem is widely recognized as an unsolved challenge in the field of NLP: even state-of-the-art models can present false information with high confidence, misleading users. For example, a model might convincingly "invent" a non-existent fact or mix up biographical details of a famous person. Since such factual errors are difficult to detect without detailed verification, researchers are actively developing methods to reduce the number of hallucinations in LLM responses.

Steps of the CoVe Method

The Chain-of-Verification method is implemented in four sequential steps:

1. Generate Baseline Response. The model, without special instructions, generates an initial response to the original query (a baseline response hypothesis). This draft response serves as a starting point and may contain hallucinations that will be identified in the subsequent steps.
2. Plan Verification Questions. Given the original query and the generated response, the model formulates a list of clarifying questions to check the factual correctness of the statements in the baseline response. These verification questions target the key facts in the response and help identify potential errors or inaccuracies.
3. Execute Verification. Next, the model sequentially and independently answers each of the formulated verification questions, trying not to rely on the initial response to avoid bias. The resulting answers are compared with the original response to detect contradictions or errors, thereby identifying parts of the baseline response that are not factually supported.
4. Generate Final Response. Finally, based on the discrepancies found, the model generates a revised, final response. This response incorporates corrections based on the verification results, which improves its factual accuracy and reduces the likelihood of hallucinations.

Each of these stages is performed through additional prompts to the same LLM, but with different instructions. That is, the model successively acts as a responder, then as a verifier (asking and answering questions), and finally as an editor of the final output.

Verification Implementation Variants

The authors of the method tested several variants for implementing the verification step, which differ in how the verification questions are asked and answered:

• Joint. The model generates both the verification questions and their answers within a single prompt. This option is less preferred, as the model, by answering immediately, may hallucinate facts and repeat errors from the original response due to bias.
• 2-Step. The verification questions are first generated in a separate prompt, and then in the next prompt, the model answers the formulated list of questions. Separating these stages helps to avoid the influence of the initial response when generating questions.
• Factored. The model answers each verification question separately using multiple sequential prompts (one per question). This approach prevents simple copy-pasting of fragments from the original response: the answers to the verification questions are formulated autonomously, which reduces the risk of repeating the initial hallucination. A disadvantage is the increased computational cost, as the number of prompts grows proportionally with the number of questions.
• Factored + Revise. After receiving answers to all verification questions, the model performs an additional comparison and revision step. Using a separate prompt, it compares the obtained facts with the original response and explicitly notes discrepancies, after which it generates the final, corrected answer. This additional step forces the system to more carefully analyze the differences and integrate the corrected information into the final output.

Experimental Results

The Chain-of-Verification method was tested on a range of tasks sensitive to factual accuracy. These included questions requiring the listing of facts from a knowledge base (lists from Wikidata and Wikipedia categories), questions requiring multiple answers from different parts of a text (MultiSpanQA), and long-form text generation tasks (e.g., biographies).

The results showed a significant reduction in hallucinations across all task types when using CoVe compared to baseline models without self-verification. The "factored + revise" variant proved to be particularly effective, delivering the best accuracy scores. For instance, in the biographical text generation task, applying CoVe to the LLaMA-65B model (a 65-billion-parameter LLM) increased its FactScore from ~63.7 to ~71.4 points. This increase in FactScore indicates that the final responses contained more verified facts and fewer fabricated details.

Furthermore, an LLM enhanced with the verification chain was able to outperform even some more powerful or specially equipped systems. For example, LLaMA-65B with CoVe achieved a higher FactScore than ChatGPT (an OpenAI model) and surpassed Perplexity.ai—a model augmented with web search for factual support. This is noteworthy because Perplexity uses external sources for information retrieval, whereas CoVe improves quality by relying solely on the model's internal reasoning and self-verification capabilities. However, for very rare facts (requiring highly specific knowledge), retrieval-augmented systems like Perplexity still hold an advantage, but on the majority of questions, CoVe provided more accurate answers.

Limitations and Future Work

It should be noted that while Chain-of-Verification significantly reduces the rate of hallucinations, it cannot eliminate them entirely. The model can still make mistakes if the verification questions fail to cover an incorrect detail or if the LLM itself does not know the correct fact. Additionally, CoVe increases the computational load: a single user query requires multiple sequential calls to the model (generating the response, generating questions, answering them, and final assembly), whereas a standard model responds in a single step. Nevertheless, the authors show that the total cost of CoVe is comparable to other multi-step hallucination detection approaches and remains a practical solution.

In their paper, the Meta AI researchers suggested possible directions for improving the method. One obvious path is to combine CoVe with the use of external tools, such as integrating a web search module or knowledge bases at the verification stage. This would allow for the retrieval of reliable external information to more robustly confirm or refute facts from the initial response. Thus, Chain-of-Verification represents a step toward more responsible and accurate AI systems: it demonstrates that by compelling a model to critically re-examine its own response, its quality can be substantially improved, and the spread of fabricated facts in generated text can be reduced.

Links

• Original paper "Chain-of-Verification Reduces Hallucination in Large Language Models" on arXiv
• Paper "Chain-of-Verification Reduces Hallucination in Large Language Models" in the ACL Anthology
• Chain of Verification (CoVe) — Understanding & Implementation — article on Medium

Bibliography

• Dhuliawala, S. et al. (2023). Chain-of-Verification Reduces Hallucination in Large Language Models. arXiv:2309.11495.
• Manakul, P. et al. (2023). SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. arXiv:2303.08896.
• Yang, B. et al. (2025). Hallucination Detection in Large Language Models with Metamorphic Relations. arXiv:2502.15844.
• Liang, X. et al. (2024). Internal Consistency and Self-Feedback in Large Language Models: A Survey. arXiv:2407.14507.
• Lightman, H. et al. (2023). Let’s Verify Step by Step. arXiv:2305.20050.
• Ling, Z. et al. (2023). Deductive Verification of Chain-of-Thought Reasoning. arXiv:2306.03872.
• Lyu, Q. et al. (2023). Faithful Chain-of-Thought Reasoning. arXiv:2301.13379.
• Madaan, A. et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651.
• Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903.
• Wang, X. et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171.
• Yao, S. et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv:2305.10601.