Reflection, Reflexion, and Self-Refine: Iterative Self-Correction

The family

Self-Refine

Madaan et al. 2023 introduced Self-Refine as a single-model iterative improvement loop. The model generates output, then prompts itself with a critique prompt to identify weaknesses, then generates a revised output incorporating the critique. The loop continues until the model judges further refinement unnecessary or an iteration cap is hit.

The original paper reports gains across seven tasks (math, code optimisation, dialogue response, sentiment reversal, acronym generation, code readability, constrained generation), with the largest improvements on tasks where a clear constraint or rubric exists and the smallest on open-ended generation tasks.

Reflexion

Shinn et al. 2023 extended the idea to multi-episode learning. After each task attempt, the agent verbalises what went wrong (a self-reflection in natural language) and stores that reflection in an explicit memory. On the next attempt of the same or a similar task, the prior reflections are added to the prompt as context.

The framing in the paper is verbal reinforcement learning: the reflection memory plays the role of a learned policy, but encoded as natural-language text rather than gradient updates. The advantage is that the lesson is human-readable and immediately usable; the limitation is that the “policy” is whatever the model can articulate about its own failures, which is not always the same as the actual cause.

Generic reflection (framework-level)

Beyond the named papers, framework documentation uses “reflection” loosely to mean any outer critique loop: a separate LLM call that judges the inner agent’s output, an automated test that checks the result, a human-review step that gates the next iteration. LangGraph’s reflection examples and CrewAI’s reviewer-role pattern both implement this generic shape.

The common structure

All three variants share four steps: produce a candidate output, critique it (same model, separate model, deterministic check), revise the output incorporating the critique, and iterate until convergence or budget exhausted.

When it helps and when it does not

The empirical record across the source papers and follow-up work points to a clear pattern. Reflection helps most on tasks with three properties: a clear external signal exists (tests pass, a schema validates, a constraint is met or not met); the model is good at recognising the failure but struggled to produce the right output first time; and the iteration budget is generous enough for two or three rounds.

Reflection helps least on open-ended generation tasks (creative writing, persuasive argument), tasks where the model’s critique is as miscalibrated as its initial output (the “blind leading the blind” failure), and tasks where the cost of an additional iteration exceeds the value of the marginal quality improvement.

The cost problem

Each reflection iteration is one or more additional LLM calls. A three-iteration reflection loop with a separate critic model multiplies token cost by roughly four (the original generation, the critic call, the revised generation, repeated). On long reasoning tasks this matters. Production deployments almost always cap iterations at two or three and add an early-exit condition that stops the loop when the critique reports no significant issues.

The cost-quality trade-off is explicit and worth measuring on the actual task: many benchmarks see most of the quality lift in the first revision, with diminishing returns after.

Composition with other patterns

Reflection composes naturally with both ReAct and plan-and-execute. The most common production architecture wraps a reflection loop around a ReAct inner loop: ReAct produces an answer through interleaved reasoning and acting; the reflection layer judges the final answer and either accepts it or sends the agent back. For plan-and-execute, reflection often sits between the plan and the execution, judging the plan before any action is taken; the agent re-plans if the critique flags issues.

See the limitations reference for the autonomy implications: a reflection layer is one of the cheapest ways to insert a safety check before destructive actions.