AI Agents — Engineering Implementations

ReAct and the Birth of Agent Memory: Why Reasoning Is Memory Creation

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.

ICLR 2023

A reinterpretation of ReAct through the lens of memory systems. Most engineers see ReAct as reasoning plus acting. The deeper contribution is that it introduced explicit working memory into LLM agents — every thought is a state update, every observation is a memory write, and the evolving context becomes the foundation for all modern agent memory architectures from MemGPT to Letta to SimpleMem.

First paper to introduce working memory into LLM agents via accumulated thought-action-observation context
Every reasoning trace is a state transition — thoughts are memory writes, not just text generation
Augmented action space A-hat = A union L treats language itself as a state-modifying operation
Direct intellectual ancestor of MemGPT, Letta, SimpleMem, Mem0, and all modern agent memory systems
71% success on ALFWorld vs 37% for agents trained on 100,000+ expert demonstrations — memory enables this

AI Agents Memory Systems Working Memory ReAct Tool Use Agent Architecture State Management Production Systems MCP LangGraph DSPy Amazon Bedrock Tool Calling

Read Implementation → View Original Paper ↗

Toolformer: How a Language Model Teaches Itself to Use Tools

Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., Scialom, T.

NeurIPS 2023 · arXiv 2302.04761

An engineering dissection of Toolformer — a language model that teaches itself when and how to call external APIs (calculator, QA, search, translation, calendar) in a self-supervised way, with no human tool-use annotations. Covers the sample-execute-filter-finetune loop, the perplexity-based filtering criterion that decides which API calls are actually useful, the API-call token format that needs no vocabulary change, inference-time API interruption, the finding that tool use emerges only above ~775M parameters, and the honest limits: no chained or interactive tool use.

Teaches tool use self-supervised: a model annotates a plain-text corpus with candidate API calls, keeps only the ones that measurably help predict the following tokens, and fine-tunes on the result — no human tool-use labels
The filtering criterion is a loss comparison: keep an API call only if seeing its input AND result lowers cross-entropy on future tokens by at least a threshold versus not calling or calling blind
API calls are encoded with ordinary tokens ([, ], ->) so no vocabulary change is needed, and at inference the decoder is interrupted at the -> token to run the tool and splice in the result
Tool use is emergent: a 6.7B GPT-J beats the 25x-larger GPT-3 175B on factual and math tasks, but the ability to use tools well only appears above ~775M parameters

Toolformer Tool Use Function Calling Self-Supervised LLM Agents APIs In-Context Learning GPT-J Augmented LMs

Read Implementation → View Original Paper ↗

LLM-Based Agents: The Brain–Perception–Action Framework

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (Fudan NLP Group)

arXiv 2309.07864

An engineering dissection of 'The Rise and Potential of LLM-Based Agents: A Survey.' Instead of summarizing 86 pages, it extracts the one thing an engineer can build on: the Brain–Perception–Action framework that decomposes an LLM agent into a controller (the LLM), an input space (perception), and an output space (action). Covers the perceive-think-act loop, the brain's memory and reasoning/planning typology, the multi-agent taxonomy (disordered vs ordered cooperation, adversarial debate), the two human-agent paradigms (instructor-executor, equal partnership), and the open problems that still gate real deployment.

Decomposes any LLM agent into three modules — Brain (the LLM controller: memory, knowledge, reasoning, planning), Perception (expands the input space beyond text to multimodal), and Action (expands the output space to tools and embodied action)
Frames the agent as a perceive-think-act loop that continuously takes feedback from its environment — the same loop ReAct, Toolformer, and tool-calling systems each implement a slice of
Organizes multi-agent systems into a clean taxonomy: cooperative (disordered free-for-all vs ordered pipelines like MetaGPT) and adversarial (debate and self-play that sharpen answers)
Names the failure modes that matter in production — cascading hallucination amplified by agent-to-agent interaction, the virtual-to-physical gap, and the difficulty of coordinating many agents into genuine collective intelligence

LLM Agents AI Agents Agent Architecture Multi-Agent Systems Perception Planning Memory Tool Use Survey Agent Fundamentals

Read Implementation → View Original Paper ↗

LLMCompiler: Why DAG Planning Is the Future of Tool-Calling AI Agents

Kim, S., Moon, S., Tabrizi, R., Lee, N., Mahoney, M. W., Keutzer, K., Gholami, A.

ICML 2024

A compiler-inspired orchestration framework that automatically identifies parallelizable LLM function calls, executes them concurrently via a DAG-based planner, and delivers up to 3.7x lower latency and 6.7x cost reduction over sequential ReAct-style execution. This architectural shift from sequential reasoning to parallel compilation fundamentally changes how production AI agents should orchestrate tool use.

Reduces multi-tool agent execution from O(N) sequential LLM calls to 2 LLM calls regardless of task count
DAG-based planning with dependency resolution enables true parallel tool execution without sacrificing correctness
Task Fetching Unit provides LLM-free scheduling — pure orchestration logic with zero inference cost
3.74x latency speedup and 6.73x cost reduction on Movie Recommendation vs ReAct
Dynamic replanning loop handles workloads that cannot be statically compiled upfront

AI Agents Tool Use Parallel Execution DAG Planning LLM Orchestration Function Calling Production Systems Compiler Theory

Read Implementation → View Original Paper ↗

SimpleMem: Efficient Lifelong Memory for LLM Agents — Semantic Lossless Compression, Synthesis & Intent-Aware Retrieval

Liu, J., Su, Y., Xia, P., Han, S., Zheng, Z., Xie, C., Ding, M., Yao, H.

arXiv 2601.02553

An engineering dissection of SimpleMem — how semantic lossless compression, online synthesis, and intent-aware retrieval give LLM agents efficient lifelong memory at 30x lower token cost.

Reframes agent memory as a three-stage pipeline: Semantic Structured Compression → Online Semantic Synthesis → Intent-Aware Retrieval Planning, built on 'semantic lossless compression'
Distills raw dialogue into multi-view indexed memory units (semantic embedding + lexical keywords + symbolic metadata) with pronouns resolved and time anchored to absolute ISO-8601
Reports +26.4% average F1 over Mem0 on LoCoMo while cutting inference-time token consumption up to 30x versus full-context serving
Maps every stage to the real released code (aiming-lab/SimpleMem): MemoryBuilder, HybridRetriever, the MemoryEntry schema, LanceDB + Tantivy storage — and flags where the paper's formalism and the shipped engine diverge

SimpleMem Agent Memory Lifelong Memory LLM Agents Semantic Compression Memory Consolidation Adaptive Retrieval Hybrid Retrieval RAG LoCoMo Token Efficiency

Read Implementation → View Original Paper ↗