Self Improving AI Agents

Agents that plan, search, and refine their own reasoning — tree search, multistep planning, self-reflection, and the feedback loops that let LLM agents improve over a task.

LATS: Language Agent Tree Search for Multistep Planning and Reasoning

LATS: Language Agent Tree Search for Multistep Planning and Reasoning

Zhou, A., Yan, K., Shlapentokh-Rothman, M., Wang, H., Wang, Y-X.

ICML 2024

An engineering-first breakdown of Language Agent Tree Search (LATS): how MCTS, environment feedback, an LM value function, self-reflection, and memory wrap a frozen LLM into a self-improving planner — with runnable code in LangChain, DSPy, and LangGraph for AI-driven drug discovery.

  • LATS wraps a pretrained LM with an inference-time search-and-feedback loop — no new model is trained and no weights are updated
  • Six operations — selection, expansion, evaluation, simulation, backpropagation, reflection — turn a single-shot agent into a planner over a tree of trajectories
  • The value function V(s) = λ·LM(s) + (1−λ)·SC(s) reuses the LM as its own evaluator, avoiding a trained value network
  • Self-reflection injects a semantic gradient — a written diagnosis of failure — that is far richer than a scalar reward and needs no gradient descent
  • Full worked implementation for AI drug discovery: LangChain for the tool environment, DSPy for optimizable cognition, LangGraph for the MCTS control loop
Self-Improving Agents Multistep Planning Reasoning Monte Carlo Tree Search MCTS In-Context Learning Self-Reflection Inference-Time Search LATS LangGraph DSPy LangChain Drug Discovery Agent Architecture
Read Implementation → View Original Paper ↗