Fixing LLM Agent Failures Through Structured Plan Review

Recent field tests examining autonomous agent workflows indicate that a major source of system failure stems from faulty planning rather than poor execution. Many developer teams focus heavily on improving execution layers such as tools, memory, and retrieval augmented generation, while neglecting the initial phase where an agent breaks down a high level goal. When an agent generates a sequence of actions in a single hidden chain of thought, it often misses critical preconditions, ordering constraints, or rollback paths. A plan may appear coherent at a glance while containing structural flaws that lead to severe production incidents once the agent begins mutating state.

To investigate this issue further, developer Debashish Ghosal built a system called PlannerCritic, which treats agent plans like software pull requests. The engine separates responsibilities by having one language model draft the plan, another review it, and a set of deterministic gates check the structural integrity. The planner must revise its steps until the workflow reaches a state that is deemed safe enough to approve or specific enough to require human escalation. Ghosal tested this architecture using 157 scenarios spanning 35 different domains, including databases, Kubernetes orchestration, identity management, messaging, and blockchain recovery.

The field test results revealed distinct operational modes based on the risk tolerance configured in the engine. Balanced mode treated model findings as advisory warnings while using deterministic code gates as a hard floor for safety, and these plans consistently passed review. Strict mode functioned as an adversarial filter designed to reject any plan that was not completely clean, and it produced zero passes across the tested scenarios. This divergence demonstrated that the engine was successfully executing its rules rather than failing arbitrarily, highlighting that risk tolerance is a core product decision for developers building agentic workflows.

Analyzing the strict mode failures uncovered 132 concrete blockers concentrated in three main categories, namely missing preconditions, topological ordering errors, and inadequate rollback coverage. Upgrading the primary planning model to GPT-4o did not resolve these structural issues, proving that the challenge is not simply a limitation of smaller models. The data showed that language models can readily describe individual tasks, but they struggle to reliably close preconditions, enforce valid execution order, and scope rollbacks correctly. Consequently, the most effective remediation involves deterministic post generation validation rather than relying entirely on model scale.

Implementing strict rules also required refining the evaluation contract itself. Initially, the reviewing model acted as an adversarial agent that blocked plans for lacking thoroughness rather than just violating safety parameters, which caused unnecessary escalations. Adding code guardrails ensured that advisory feedback was separated from hard safety violations, allowing the system to ignore prompt injection attempts embedded in malicious goals and escalate them appropriately. These adjustments emphasize that developers should treat severity as an explicit API contract enforced by code rather than leaving interpretation up to the model.

The findings suggest several practical takeaways for teams building AI agents. Developers should treat plans as inspectable artifacts rather than hidden reasoning chains, separate the planning role from the reviewing role, and enforce non-negotiable constraints through deterministic code checks. Testing planning engines across diverse corpora rather than single happy path demos reveals hidden architectural vulnerabilities. Ultimately, proper planning acts as the first real safety boundary for autonomous systems, ensuring that powerful orchestration tools do not simply allow agents to fail faster.