Self-Refine Prompting

Self-Refine is an approach in prompt engineering that allows large language models (LLMs) to iteratively improve a generated response based on their own feedback. The idea was proposed by a group of researchers led by Aman Madaan in 2023 and is based on the observation that, much like humans, language models do not always produce the best result on the first attempt.

In this method, the same LLM sequentially performs three roles:

1. Generator: Creates an initial draft response to the prompt.
2. Critic (Feedback): Evaluates its own response and provides constructive feedback.
3. Refiner: Uses this feedback to create an improved version of the response.

This iterative "generate → feedback → refine" cycle can be repeated multiple times until the desired quality is achieved or a stopping condition is met.

Importantly, Self-Refine does not require additional model training, fine-tuning, or external data—the entire process is managed exclusively through prompts at the inference stage.

Implementation Mechanism

The Self-Refine method is implemented through a sequence of specially crafted prompts that guide the model's behavior.

1. Initial Generation. The model receives the original prompt and generates a draft response.
2. Feedback Generation. The model is instructed to analyze its own previous response and identify its shortcomings. For example, it might note that the response is not detailed enough or contains a logical error. The result is textual feedback with specific comments and recommendations.
3. Iterative Refinement. The model receives a new prompt containing the original query, its initial response, and the generated feedback. Based on this, it creates an improved version of the response.

This two-step "critique → refine" cycle can be executed multiple times. The context for each new iteration includes previous versions of the response and the feedback, which helps the model avoid repeating mistakes. Techniques like few-shot prompting are often used to guide the model's behavior, where the prompt includes examples of the desired format for feedback and corrections.

Effectiveness and Applications

The Self-Refine method has demonstrated its effectiveness on a range of tasks that require repeated refinement, such as:

• Generating conversational responses.
• Creative story continuation.
• Solving mathematical problems with step-by-step reasoning.
• Optimizing program code.

In the original study, responses obtained using Self-Refine were, on average, ~20% more preferred by human evaluators and automatic metrics compared to single-step generation. Improvement was achieved even for state-of-the-art models like GPT-4, indicating that even powerful LLMs often just need an additional step of reflection to correct their own mistakes.

Similar approaches, such as RCI (Recursive Criticism and Improvement), have also shown high effectiveness in interactive tasks, for example, in computer control and solving logical problems. Combining RCI with the Chain-of-Thought technique produced a synergistic effect, significantly improving the model's ability to solve complex problems by incorporating a self-verification step.

Limitations and Current Research

Despite promising results, research shows that self-iterative improvement has several limitations.

• Self-bias: Models struggle to judge their own responses impartially. LLMs tend to view their own generated text favorably and evaluate it with insufficient criticism, which can lead to stagnation or even a decrease in quality after several iterations.
• Overconfidence: It has been observed that as the number of self-correction iterations increases, the model can become overconfident in its answers, even if they have not become more accurate. This leads to an increase in the Expected Calibration Error (ECE)—a discrepancy between the model's confidence and its actual accuracy.
• Computational Cost: The method requires multiple calls to the LLM to obtain a single final answer, which significantly increases latency and cost compared to single-pass generation.

Current research is aimed at addressing these problems. One direction is the implementation of confidence calibration mechanisms at each iteration step. Another is the use of external information sources or tools (e.g., code execution, data retrieval) for more objective self-assessment.

Comparison with Other Techniques

• Chain-of-Thought (CoT): CoT focuses on generating a linear chain of reasoning to arrive at an answer. Self-Refine, in contrast, focuses on iteratively improving an already generated answer (which may itself have been produced using CoT).
• Tree of Thoughts (ToT): ToT explores multiple parallel reasoning paths in the form of a tree, whereas Self-Refine improves a single path iteratively. ToT is a technique for exploring the solution space, while Self-Refine is a technique for optimizing a specific solution.

Links

• Official Self-Refine project website
• Original research paper "Self-Refine: Iterative Refinement with Self-Feedback"

Further Reading

• Madaan, A. et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651.
• Kim, G. et al. (2023). Language Models Can Solve Computer Tasks. arXiv:2303.17491.
• Gou, Z. et al. (2023). CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. arXiv:2305.11738.
• Huang, J. et al. (2023). Large Language Models Cannot Self-Correct Reasoning Yet. arXiv:2310.01798.
• Pan, L. et al. (2023). Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Self-Correction Strategies. arXiv:2308.03188.
• Jiang, C. et al. (2024). Importance Weighting Can Help Large Language Models Self-Improve. arXiv:2408.09849.
• Liang, X. et al. (2024). Internal Consistency and Self-Feedback in Large Language Models: A Survey. arXiv:2407.14507.
• Zhu, D. et al. (2025). Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models. arXiv:2504.02902.
• Hao, Q. et al. (2025). RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning. arXiv:2505.14140.
• Cui, Y. et al. (2023). Check Your Facts and Try Again: Improving Large Language Models by Reducing Hallucination. arXiv:2302.12813.
• Wei, Z. et al. (2024). ReSearch: Iterative Self-Reflection for Better LLM Calibration. arXiv:2405.13022.