AI Agents Are Distributed State Machines — What That Means for How You Build Them

Most tutorials on AI agents start with the prompt. Here is your system message. Here are your tools. Here is your loop. Go build. But this framing misses something fundamental about what you are actually constructing when you wire multiple agents together.

Nicole Königstein, CEO of Quantmate — an agentic quant research environment applying transformers and multi-agent systems to financial markets — starts from a different place entirely. She is authoring the O’Reilly definitive guides on both transformers and AI agents, and in the opening chapters of the agents book, she draws a parallel that reframes the entire design conversation.

“In my mind, agents are distributed state machines,” Königstein says. “You have that fixed vocabulary and the fixed states and decision points where you are in. They have a certain amount of actions, so tools, interactions they can do.”

Why the state machine framing matters

Finite state machines have a property that most agent tutorials ignore: they have a bounded set of states, transitions, and actions. When you build a multi-agent system, you are defining — whether you realize it or not — what each agent can do, what states it can be in, and how it transitions between them.

The “distributed” part is where architecture decisions get interesting. If your agents are distributed state machines, then the topology — how they are arranged and how they communicate — determines the system’s behavior at least as much as what any individual agent can do.

Königstein maps agent topologies to organizational structures that make the tradeoffs immediately visible. A centralized topology works like a team with a tech lead: one orchestrator knows the goal and delegates to specialist workers. A blackboard topology works like a creative brainstorming room: independent agents share a common workspace, contributing ideas and building on each other’s output.

“If you have centralized, you have that kind of tech lead within the programming team, and then you have your sub agents, all the programmers, and they are the workers,” she explains. “But maybe then you have a completely different organization — a creative agency — so you have those people sitting in the room and do brainstorming together, and you have that blackboard or whiteboard they are writing at.”

The memory problem topologies create

Each topology creates a distinct memory management challenge — and this is where many multi-agent systems silently fail.

In a centralized topology, the supervisor needs enough context to make routing decisions. But does it need the full local state of every sub-agent? Probably not, and loading all that context burns tokens on information the supervisor will never use.

“Not every agent involved in that system needs to have every information about what the other agent is doing,” Königstein notes. “You need to decide — does every agent need to have the local state and memory of that specific agent, and how do you orchestrate then how they communicate.”

In a blackboard topology, the problem inverts. Agents benefit from shared state — that is the whole point. But shared state grows, and managing that memory growth while keeping token consumption under control becomes its own engineering challenge. The whiteboard fills up, and the agents spend more time reading it than contributing to it.

From topology to team dynamics

Königstein pushes the organizational metaphor further when discussing agent collaboration. If agents are team members, then the question isn’t just what each one does — it is how they learn to work together.

“One thing what you would want to consider is maybe it might even make sense if it needs to be very reliable — it makes sense to teach them actually to collaborate better,” she says. “And this can be done with reinforcement learning.”

This is not fine-tuning for domain knowledge. It is reinforcement learning applied to coordination behavior — training the underlying models on full task trajectories so they learn more efficient collaboration patterns. The output of the training is not a better prompt. It is updated model weights that change how an agent reasons about handoffs, delegation, and when to ask for clarification versus proceeding independently.

The distinction matters because it means the agent’s behavior changes at the level of the model, not the level of the prompt. A well-trained team of agents with mediocre prompts can outperform a poorly coordinated team with perfect prompts.

Designing from roles, not from agents

Königstein’s practical advice for builders starts before a single agent exists. She asks companies to describe the skills and roles their workflow requires, then maps backward to agents and topology.

“Tell me what kind of skills they want to do, and from there you can decide — what kind of tasks are involved and how do you break it down.”

When she walked through designing a patent filing system live, the first move was not choosing a model or writing a system prompt. It was identifying two teams — research (web search, patent API access, synthesis) and writing (drafting, fact-checking, citation validation) — and then deciding how those teams should communicate.

The system design followed from the role map, not the other way around. The agents, the topology, and the memory architecture all fell out of understanding what the work actually required. That sequence — roles first, agents second — is the state machine thinking in practice.

FAQ

What does it mean that AI agents are distributed state machines?

Each agent operates within a bounded set of states, actions, and transitions — like a finite state machine. When multiple agents work together, they become distributed state machines where the topology (how they communicate and share state) determines system behavior as much as individual agent capabilities. Nicole Königstein uses this framing in her O’Reilly guide on AI agents.

What are the different topologies for multi-agent systems?

Centralized (one orchestrator delegates to workers, like a tech lead managing programmers), decentralized (agents coordinate peer-to-peer), and blackboard (agents share a common workspace for collaborative reasoning). Each topology creates different communication patterns, memory requirements, and failure modes. Königstein recommends choosing based on the specific coordination needs of your task.

How does agent topology affect memory management?

Centralized topologies require the supervisor to carry enough context for routing decisions — but not necessarily every sub-agent’s full state. Blackboard topologies benefit from shared memory but face growth problems as the shared workspace fills up. Königstein emphasizes that deciding what context each agent needs is one of the most impactful architecture decisions you make.

Can you train AI agents to collaborate better with each other?

Yes, through reinforcement learning applied to full task trajectories. Königstein distinguishes this from fine-tuning for domain knowledge: RL teaches behavioral reasoning — how to coordinate, delegate, and handle handoffs — by actually updating model weights. The result is agents that learn more efficient collaboration patterns, not just better prompts.

What is the difference between reinforcement learning and fine-tuning for AI agents?

Fine-tuning with supervised learning teaches domain knowledge — classifying inputs, mapping to specific outputs where there is one correct answer. Reinforcement learning teaches behavioral reasoning — how to act under open-ended conditions where no single answer is correct. Königstein uses RL to train agents on multi-step task trajectories, updating model weights for better reasoning paths.

How should I start designing a multi-agent system?

Map the roles and skills your workflow requires before deciding on agents or architecture. Königstein consults for companies by asking what tasks are involved, what needs verification, and how sensitive the output is. The agent count, topology, and memory architecture should follow from the role map — not the other way around.

What is a blackboard architecture in multi-agent systems?

A topology where agents share a common workspace — like a brainstorming room with a whiteboard. Agents contribute independently, read each other’s contributions, and build on shared ideas. Königstein notes this works well for creative or research tasks but creates memory management challenges as the shared state grows with every agent contribution.

Why do organizational metaphors help explain multi-agent design?

Because the coordination problems are structurally identical. A centralized agent topology has the same bottleneck risk as a team with one overloaded manager. A blackboard topology has the same meeting-overhead problem as brainstorming sessions that never converge. Königstein uses org chart analogies to make topology tradeoffs immediately intuitive for builders.

What is the first question to ask before building a multi-agent system?

Does this task actually require multiple agents? Königstein starts every project by identifying roles and tasks, then evaluates whether the coordination overhead of multiple agents is justified. Many tasks that seem to require multi-agent coordination can be handled by sequential model calls with structured output — simpler, cheaper, and with fewer failure points.