A 2024 paper titled "Large Language Models Cannot Self-Correct Reasoning Yet" found that LLMs don't reliably catch their own mistakes without external input, and sometimes make outputs worse by talking themselves into believing wrong answers are correct—a phenomenon called the coherence trap. Self-correction only works when agents have an external signal to check against, rather than relying on the model's own judgment.
This tutorial demonstrates how to build a code-generation agent with a test-based verifier, a bounded retry loop, and a structured escalation path. It also covers adding a consistency-based confidence gate that generates an independent second solution to confirm correctness before proceeding.
The Stanford Reflexion paper showed that agents with verbal self-reflection achieved 91% pass@1 on HumanEval, up from an 80% baseline—a 20-point improvement. The tutorial requires Python 3.10+, an Anthropic API key, and installs langgraph, langchain-anthropic, pytest, and python-dotenv.