AI Chatbots vs. AI Agents: What Actually Changes?

The two words are used as though they describe two levels of sophistication — a chatbot is the simple one, an agent is the clever one. That framing is wrong in a way that costs money, because it suggests the upgrade path is a better model. It usually is not.

A chatbot and an agent can be built on the same model, with the same prompt, by the same team. What separates them is what the system is permitted to do when it has finished thinking.

The line is the output, not the intelligence

A chatbot's output is text. Someone reads it, judges it, and decides what to do. The person is the actuator: nothing in the business changes until they change it. That is also the safety property — a bad answer is absorbed by the reader before it reaches anything.

An agent's output is a state change. A record is written, a ticket is moved, a refund is issued, a message goes out. The model's decision reaches a system directly, and the person is now a reviewer of consequences rather than a filter on suggestions.

Everything else follows from that one difference. It is why an agent is not a chatbot with a longer prompt, and why the hard parts of building one are almost entirely outside the model.

What changes the moment a system can act

Four things become mandatory that a chatbot never needed. Each is ordinary engineering, and each is where agent projects actually fail.

Tools

An agent reaches the world through a fixed set of functions you wrote: refund_order, create_ticket, search_inventory. The set is the agent's entire vocabulary of action, so its design is a product decision, not a plumbing detail. Tools that are too coarse (update_record with a free-form payload) hand the model the ability to do things you never considered. Tools that are too fine make every task a twenty-step plan with twenty chances to lose the thread.

The useful discipline: a tool should be something you would be comfortable seeing in an audit log a hundred times a day, with arguments you can validate before you execute it.

Permissions

A chatbot inherits the permissions of whoever is typing. An agent runs on its own, often against systems where it holds broad credentials because that was the quickest way to make it work. That is the failure waiting to happen — not a dramatic one, usually just an agent acting correctly on a record it should never have been able to see.

Scope the agent's credentials to the task, not to the integration. If it handles refunds under a threshold, it should be unable to issue one above it — enforced by the system that holds the money, not by an instruction in the prompt.

A definition of done

A chatbot stops when it has replied. An agent has to decide when the task is finished, and that decision cannot be left to the model's judgement alone. Without an explicit stopping condition and a step budget, the common failure is not a rogue agent — it is a stuck one, retrying a call that will never succeed until something times out.

A record

When a person does the work, the trail is implicit: they remember, and their colleagues saw. When an agent does it, the only evidence is what you chose to write down. The run log — inputs, each tool call and its arguments, each result, the final state — is not observability tooling bolted on afterwards. For anything touching money, customers or compliance it is part of the product, and it is what makes the system defensible when someone asks why a decision was made.

The loop is the architecture

Strip an agent to its shape and it is a loop with a budget. The model proposes an action, the system validates and executes it, the result comes back, and the loop runs again until a stopping condition is met or the budget is spent.

// The whole pattern. The interesting decisions are all in the guards.
for (let step = 0; step < MAX_STEPS; step++) {
  const proposal = await model.next(transcript);

  if (proposal.kind === "done") return finish(proposal);

  // Not optional: the model proposed this, it did not authorise it.
  const check = policy.check(proposal.tool, proposal.args, actor);
  if (!check.allowed) return escalate(check.reason);

  const result = await tools.run(proposal.tool, proposal.args);
  await runLog.append({ step, proposal, result });   // before anything else
  transcript.push(result);
}

return escalate("step budget exhausted");

Note what is missing: nothing here makes the agent smarter. policy.check runs outside the model because a permission the model can talk itself past is not a permission. runLog.append happens immediately after the call, not at the end of the run, because a crash halfway through must not erase the record of an action that already happened in the outside world.

Where the boundary genuinely blurs

The categories are not clean, and pretending otherwise leads to arguments about labels instead of decisions. A support assistant that looks up an order and drafts a reply for an agent to send is doing tool use, but every consequence still passes through a person. A system like that has an agent's architecture and a chatbot's risk profile, which is often exactly the right trade.

The question worth asking is not what to call it. It is: what is the smallest action this system can take without a person, and what happens if it takes that action wrongly a hundred times before anyone notices? The answer tells you how much of the machinery above you actually need.

Deciding which one you need

Before choosing an architecture, answer these about the specific task — not about the category of work, and not about the department.

1. Does the task have a definition of done that a program can evaluate? If finished means someone is satisfied, a person is still in the loop and you are building an assistant.
2. Is there a real action at the end? If the output is advice, an agent adds cost and risk and buys nothing.
3. Can the action be reversed, or at least held for review? Reversible actions are where agents should start; irreversible ones are where the review step earns its keep.
4. How often does the task hit a case the rules do not cover? A high exception rate is a process problem, and automating it will not fix it.
5. Would you be able to explain a specific run afterwards? If the answer depends on remembering, you need the log before you need the agent.

What agents do not change

They do not remove supervision. They relocate it — from watching the work to reviewing what the work did, which is a different job requiring different tooling. Teams that skip building that tooling end up supervising an agent by reading its output, which is slower than doing the task themselves.

They also do not fix an unclear process. An agent needs the decision rules written down more explicitly than a person ever did, and the act of writing them down is usually where the real problem surfaces — which is worth having, but is not the project anyone budgeted for. That failure has its own shape, and it is the subject of the automation trap.

The practical path is narrow and unglamorous: pick one task with a clear finish line and a reversible action, give it a small set of well-designed tools, scope its credentials tightly, log every run, and keep a person on the review queue until the log stops being interesting. That is a smaller project than most agent proposals, and it is the one that tends to survive.