Reflection & Self-Refinement
Reflection and self-refinement use feedback on an initial attempt to guide a subsequent attempt. An agent may critique its answer, interpret test results or retain lessons from a failed action, but improvement must be established by checking the revised result against an independent task criterion.
What it is
A refinement loop produces an output, obtains feedback and uses that feedback to revise the output or a later decision. Self-Refine uses a model for generation, feedback and revision without updating its weights. Reflexion retains verbal feedback in episodic memory to influence subsequent attempts. Both operate through context and control flow, rather than necessarily training a new model. Feedback can come from an external test, a human or the model itself, and those sources have different reliability. Reflection is therefore a reusable inference-time pattern whose value depends on whether the feedback identifies an actionable defect.
What the work involves
The practitioner defines a critique rubric, feedback source, revision budget and stopping rule. Feedback should point to specific errors or unmet requirements, and revisions should preserve already-correct parts of the result. External checks are preferable when correctness can be measured directly. A useful trace links each revision to the defect it was intended to fix. Evaluation compares the initial and revised outputs, including cases where refinement degrades the answer or merely changes wording without resolving the underlying problem.
Illustrative example
A code agent writes a parser and runs a test containing a quoted delimiter. The failure becomes concrete feedback: its splitting rule ignores quoting. The agent revises the parser and reruns both the failing test and earlier cases. In a separate prose task, a critic checks whether each requested section is present before revision. The evaluator distinguishes a repaired defect from an answer that has simply become longer or more confident.
Limits and common mistakes
A model can share its own original misconception when acting as critic, or invent a defect in a correct answer. Repeated revisions may oscillate or consume resources without progress. Verbal reflection also does not prove that a lesson will transfer to new situations. Quality requires measurable correction, a bounded loop and careful retention of useful feedback. Calling the mechanism reinforcement learning can be misleading when the procedure changes context but leaves model weights unchanged.
Prerequisites
- hardAI Agent Design
Reflection is a reusable pattern inside the agent loop.
Sources and further reading
- Self-Refine: Iterative Refinement with Self-Feedback
Defines generation, self-feedback and refinement without additional model training.
- Reflexion: Language Agents with Verbal Reinforcement Learning
Describes storing verbal feedback in episodic memory to influence later attempts rather than updating weights.
Last updated: 2026-10-10