Every LLM you've ever used is stateless. You send a prompt, get a response, and the model forgets you exist. The context window is its entire universe, and when the conversation ends, that universe vanishes.

This works fine for Q&A. It completely breaks down when you need multiple agents working together across time — planning, coding, debugging, reviewing, remembering what they tried last Tuesday. The gap between a single stateless model and a functioning multi-agent system is enormous, and closing it turns out to require ideas borrowed not from computer science, but from neuroscience.

The problem: capable amnesiacs

The current generation of multi-agent frameworks can do impressive things within a single session. MetaGPT assigns software development roles (product manager, architect, engineer, QA) and passes structured artifacts between them. AutoGen lets agents converse freely, form dynamic groups, and execute code. These systems can build working software from a single sentence prompt.

But they forget everything the moment the session ends. They can't learn from past failures. They can't build on what worked last week. Each run starts from scratch, as if the previous run never happened.

This is the core limitation. You can scale agents spatially — add more of them, give them different roles, wire them into elaborate topologies. But without temporal depth, you have a team of brilliant people with permanent amnesia. They'll solve the same problems again and again, and repeat the same mistakes, because no one remembers.

AutoGen's bet: conversation is the primitive

The most interesting architectural choice I found while studying these frameworks is AutoGen's. While MetaGPT and CAMEL use strictly static conversation patterns with specialized pipelines, AutoGen makes a different bet: everything is a conversation.

Every agent inherits from a single abstraction called ConversableAgent. That's it. An agent that can receive a message, react to it, and respond. From this one building block, you get three variants: an AssistantAgent (powered by an LLM for reasoning), a UserProxyAgent (powered by humans and code execution), and a GroupChatManager (orchestrates who speaks next).

There's no centralized control plane. Agent A sends a message. Agent B receives it, triggers a generate_reply() call, and sends back. The workflow is entirely decentralized, driven by an auto-reply loop that runs until a termination condition is met.

This matters more than it sounds. Every step is visible, traceable, and interruptible. It's not a black box. When an agent decides something, you can see why — it's right there in the conversation history. And because the human is just another conversable agent in the same protocol, human-in-the-loop isn't bolted on as an afterthought. It's native.

Three topologies, three tradeoffs

How you wire agents together determines what the system can do. I found three distinct patterns in the literature.

Static pair: two agents talking back and forth. Simple, predictable, good for focused tasks like math solving. One generates, one verifies. No orchestration overhead.

Hierarchical: a commander agent delegates to specialized workers. OptiGuide uses this — a Commander receives the user prompt, passes it to a Writer who drafts code, then a Safeguard agent checks the code for security issues before it runs. If the Safeguard rejects it, debug info goes back to the Writer for revision. This separation boosted unsafe code detection F-1 by 35%, and the total workflow codebase shrank from 430 lines to 100.

Dynamic group chat: a GroupChatManager selects the next speaker based on context. No fixed order. The system decides who should talk based on what the conversation needs right now. This is the most flexible but also the hardest to control. Speaker selection itself becomes a reasoning task.

The grounding problem: agents get stuck

One failure mode really stood out. When an assistant and executor agent pair encounter an error, they can fall into a repetitive loop — trying the same approach over and over, each time failing the same way. The agent equivalent of banging your head against a wall.

The solution from the research is a grounding agent — an observer that watches the conversation and injects commonsense rules when it detects a loop. For example: "You must find and take the object before you can examine it." Simple constraints that the planning agent should know but doesn't, because LLMs lack persistent physical reasoning.

Adding this grounding agent yielded a 15% performance gain across 134 unseen tasks. Not by making the agent smarter, but by giving it an external check against its own fixation patterns.

This is where the research on multi-agent risks connects. These systems can fail in subtle ways — infinite loops, cascading errors between agents, confidently wrong consensus where all agents agree on something false. There is a lot of active work on taxonomizing and preventing these failures. The field is moving fast, and much of it is unsolved.

The memory question: four types, and why AI needs to sleep

This was the part that clicked hardest for me. The MemAgents framework from ICLR 2026 argues that the real frontier isn't spatial (more agents) but temporal (memory across time). Without memory, scaling agents horizontally just gives you faster amnesia.

They identify three memory layers, and the analogy to human cognition is direct.

Working memory is what fits in the context window right now. It's active, immediate, and limited. When the conversation grows beyond the window, you lose information. This layer handles chunking and summarization to survive those limits.

Episodic memory records interaction logs — what happened, in what order, with what outcome. This is the agent equivalent of remembering that "last time I tried approach X on this type of problem, it failed because of Y." It enables temporal credit assignment: linking a decision now to an outcome later.

Semantic and parametric memory is the consolidated, lasting layer. Knowledge graphs, vector databases, fine-tuned weights. The stuff the agent "knows" without having to re-derive it.

The biological parallel runs deep. The raw interaction logs are noisy and high-volume, like hippocampal episodic traces. A consolidation filter abstracts those experiences into lasting policies — managing forgetting, extracting rules. The output is stored in a cortical analog: compact, high-utility knowledge in vector DBs and knowledge graphs.

This is where it gets weird. This consolidation process is the AI equivalent of sleep. During a session, the agent accumulates raw episodic traces. Between sessions, a background process reviews those traces, decides what's worth keeping, abstracts patterns, and stores them in semantic memory. The agent that wakes up for the next session is different from the one that finished the last one. It has learned.

The synthesis: what this means for my research

The final synthesis unifies everything into a single architecture: multiple conversable agents in a dynamic group chat, each connected to a shared memory subsystem with all three layers. Data flows horizontally through conversation. Context and consolidation flow vertically between agents and memory.

The argument here is that the future isn't a smarter model. It's a distributed cognitive architecture — conversational nodes operating continuously across time, backed by online consolidation pipelines.

I keep coming back to two things. First, the vehicle analogy. Imagine a self-driving car that forgets every trip it's ever taken. Every morning it rediscovers the same pothole, re-learns the same school zone, recalculates the same optimal route. That's what current multi-agent systems are. Memory is what turns a reactive system into an adaptive one.

Second, the observation that conversation being the core primitive isn't just an engineering choice. It's a transparency choice. When agents communicate through opaque internal representations, the system is a black box. When they communicate through natural language conversation, every decision is auditable. The human can see the reasoning. Can intervene. Can correct. This matters more as these systems get more capable and more autonomous.

For my thesis on resource-efficient multi-agent systems, the memory layer is now front and center. The question isn't just "how do you make agents coordinate" — that's partially solved by frameworks like AutoGen. The question is "how do you make them coordinate efficiently across time, on constrained hardware, without blowing up your memory budget?" Episodic memory grows linearly with interactions. Consolidation requires compute. Vector DBs require storage. On a single A100 that's already running inference, where does all of this fit?

That's the open problem I want to work on.

Papers & Resources

  • Wu et al. (2024) — AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation — arxiv.org/abs/2308.08155
  • Hong et al. (2024) — MetaGPT: Meta Programming for Multi-Agent Collaborative Framework — arxiv.org/abs/2308.00352
  • Guo et al. (2024) — Large Language Model based Multi-Agents: A Survey of Progress and Challenges — arxiv.org/abs/2402.01680
  • WMAC 2026 — AAAI Bridge Program on Advancing LLM-Based Multi-Agent Collaboration — multiagents.org/2026
  • MemAgents Workshop (ICLR 2026) — Memory for LLM-Based Agentic Systems — OpenReview
  • Wooldridge (2009) — An Introduction to Multi-Agent Systems, 2nd Edition — Wiley

I'm Mustapha Liaichi, an AI engineer exploring the frontier of LLM systems and autonomous agents. These notes document my research journey. Reach me at mustaphaliaichi@gmail.com