The chat window tells you very little
Two products can look like the same chat box while doing completely different work. One answers a question using a prompt and context. The other chooses a tool, inspects its result, decides what to do next and continues until a task is complete or a limit is reached.
The distinction I find useful is control flow. Anthropic’s engineering guide separates workflows with predefined paths from agents that let the model direct steps and tool usage. A chatbot can contain either architecture; an agent can run without a chat interface. The labels matter less than what the application is allowed to do.
Primary sourceA concrete support example
A customer asks, “Where is my order?” A basic assistant might explain the shipping policy. A retrieval-backed chatbot can find the correct policy clause. A tool-enabled workflow can authenticate the customer, look up that order and report its status. An agent becomes useful if the next step genuinely depends on what it discovers.
Now change the request to “Cancel it if it has not shipped.” Reading an order and changing it are different permissions. The application must verify access, enforce the cancellation policy and distinguish a proposed action from an action that actually completed. Fluent text cannot substitute for a transaction result.
Read the diagram as text
A concrete support example — A customer asks, “Where is my order?” A basic assistant might explain the shipping policy. A retrieval-backed chatbot can find the correct policy clause. A tool-enabled workflow can authenticate the customer, look up that order and report its status. An agent becomes useful if the next step genuinely depends on what it discovers.
Autonomy has an operating cost
Every additional loop adds latency, tokens and another place for a failure to happen. A model can select the wrong tool, repeat a successful action after a timeout or interpret untrusted tool output as an instruction. These are application design problems as much as model problems.
Use narrow tool contracts and explicit limits. A read-only search tool with bounded results is easier to reason about than a tool that can run arbitrary queries. A write operation needs authorization and protection against duplicate execution. Keep an observable record of tool outcomes without turning logs into a store of sensitive user text.
Start with the simplest control flow
If the sequence is known, write a workflow. If the system only needs to answer from a fixed body of documents, a retrieval pipeline may be enough. Reach for an agent when choosing the next step is the actual source of value, and when you can define what success and stopping look like.
Before shipping, test a missing record, permission denial, invalid tool arguments and a timeout after a write. Ask whether the final response accurately says what happened. “I tried to cancel the order” and “the order was cancelled” must remain distinct.
A good agent feels dependable because its authority is carefully designed. More tools and more loops do not automatically make it more useful.
The sources behind the ideas.
Research and primary documentation for the factual points above. Examples and recommendations are my interpretation.
