What is an AI agent, and how does it work?
What is an AI agent? It's software that uses an AI model to work toward a goal you give it, deciding which steps to take and carrying them out with tools such as web search, apps or code. A chatbot answers your message; an agent keeps going, checking the results of each step until the task is done or it needs your input.
The short version
- An AI agent pairs a large language model with tools, memory and the freedom to choose its own next step.
- Anthropic describes most agents as “LLMs using tools based on environmental feedback in a loop.”
- Google Cloud separates agents from AI assistants by autonomy: an assistant can recommend actions, but the user makes the decisions.
- Agents cost more to run than a single chatbot reply, and errors can compound across steps.
- Malicious instructions hidden in emails, files or websites can hijack an agent, so limit what it can access.
What is an AI agent in simple terms?
Google Cloud defines AI agents as “software systems that use AI to pursue goals and complete tasks on behalf of users.” AWS describes an agent as a software program that can interact with its environment, collect data and use that data to perform self-directed tasks that meet goals people set.
You don't tell an agent each step. You describe the outcome, and the agent works out the steps, uses the tools it has been given and checks whether each step worked.
Anthropic, which makes the Claude models, draws a line inside what it calls agentic systems. A workflow runs a language model and tools through steps a developer wrote in advance. An agent is a system where the model “dynamically” directs its own process and tool use, keeping control over how it accomplishes the task.
How does an AI agent work?
Google Cloud, AWS, IBM and Anthropic all put a large language model at the center. Google Cloud calls the LLM the “brain” of an agent, and AWS names models such as GPT and Claude as examples of the foundation model at an agent's core. Around that model sit a few other parts:
- Tools. Functions or outside services the agent can call, such as a search engine, a database, an API or a code runner. IBM says agents turn to tools including web searches, APIs and even other agents when they lack the information to finish a subtask.
- Memory. AWS says the memory module lets an agent retain information across interactions, sessions or tasks.
- Planning. The agent breaks a large goal into smaller steps. IBM notes that for simple tasks, planning isn't a necessary step.
Those parts run in a loop. Anthropic's description of a typical agent goes like this:
- Get the task. The agent starts with a command from a person or a conversation with one.
- Plan and act. Once the task is clear, it plans and works on its own, calling tools as it goes.
- Check the results. At each step it gets what Anthropic calls “ground truth” from the environment, such as a tool's output or the result of running code, to judge its progress.
- Pause or stop. It can stop for human feedback at checkpoints or when it gets stuck. The task usually ends when it's complete, and developers often add stopping conditions such as a maximum number of steps.
IBM gives a worked example: asked which week next year will likely have the best weather for a surfing trip in Greece, an agent pulls historical weather data from a database, asks a second agent that specializes in surfing what conditions matter, then combines the weather data with those conditions to make its prediction.
Agents often look up documents before they act, much like retrieval-augmented generation (RAG). The difference is that RAG retrieves information for a single answer, while an agent can also take actions and decide what to do next.
AI agent vs AI assistant vs chatbot
The terms overlap, and vendors don't use them the same way. This table is adapted from Google Cloud's comparison:
| AI agent | AI assistant | Bot | |
|---|---|---|---|
| Purpose | Performs tasks autonomously and proactively | Helps users with tasks | Automates simple tasks or conversations |
| Decisions | Can make decisions independently | Can recommend actions, but the user decides | Follows predefined rules |
| Tasks | Complex, multistep actions | Responds to requests and completes simple tasks | Basic interactions |
| Style | Proactive and goal-oriented | Reactive to user requests | Reactive to triggers or commands |
IBM frames the chatbot question a little differently. It describes nonagentic chatbots as ones without tools, memory or reasoning, which need continuous input from the user. Agentic chatbots, by contrast, can break a task into subtasks without human intervention and use their tools to fill gaps in what they know.
AI agent vs LLM
A large language model is the engine; an agent is the vehicle built around it. On its own, an LLM takes text in and produces text out, and IBM notes that it answers from its training data, with the knowledge and reasoning limits that brings. An agent wraps that model in tools, memory and a loop so it can fetch up-to-date information and act on it.
Anthropic calls this combination the “augmented LLM”: a model enhanced with retrieval, tools and memory. It's the basic building block of every agentic system the company describes.
Examples of AI agents
Google Cloud groups the agents organizations deploy into 6 categories:
- Customer agents answer questions, resolve issues and recommend products.
- Employee agents handle repetitive tasks, answer staff questions and edit or translate content.
- Creative agents generate content, images and ideas.
- Data agents analyze complex data.
- Code agents generate code and assist with software development.
- Security agents help mitigate attacks and speed up investigations.
Anthropic points to 2 of its own: a coding agent that edits many files to resolve a task description, and a “computer use” reference implementation in which Claude operates a computer to complete tasks.
Google Cloud also sorts agents by how they interact with people. Interactive partners, or surface agents, respond when you ask them something. Background agents work behind the scenes, are usually triggered by events and have little or no human interaction.
Separately, Google Cloud sorts agents by number. A single agent works on its own toward a goal. A multi-agent system has several agents that collaborate or compete, and each can use a different model.
How AI agents connect to apps: MCP
An agent is only as useful as the tools it can reach. The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to outside systems, such as files, databases, search engines and calendars. Its documentation compares it to “a USB-C port for AI applications”: a single standard way to plug in many tools.
According to the MCP site, it's supported by AI assistants such as Claude and ChatGPT, as well as development tools such as Visual Studio Code and Cursor. An example the project gives is an agent that can access your Google Calendar and Notion to act as a more personalized assistant.
Risks and limits of AI agents
More independence brings new failure modes. These are the main ones that vendors and NIST flag:
- Cost and compounding errors. Anthropic says the autonomous nature of agents means higher costs and “the potential for compounding errors.” It recommends extensive testing in sandboxed environments with guardrails.
- Hijacking. The US National Institute of Standards and Technology (NIST) says many AI agents are vulnerable to agent hijacking, a type of indirect prompt injection. An attacker hides instructions in data the agent reads, such as an email, a file or a website, to make it carry out a different and possibly harmful task.
- Judgment. Google Cloud says agents can struggle with tasks that need empathy, situations with high ethical stakes and unpredictable physical environments.
- Privacy. An agent needs access to your data and accounts to act for you. AWS says organizations that develop and operate AI agents should be aware of data privacy requirements.
If you use an agent, give it access only to the accounts and data a task needs, and review what it plans to do before it sends, buys or deletes anything. Protecting the accounts themselves matters too, so turn on two-factor authentication for anything an agent touches.
Frequently asked questions
What can an AI agent do?
It can complete multistep tasks that involve looking things up and acting on them, such as researching a question across several sources, editing code across many files or answering a customer's question from internal documents. What a given agent can do depends on the tools and permissions its developer or user gives it.
How do you build an AI agent?
Anthropic suggests developers start by calling LLM APIs directly, since many agent patterns take only a few lines of code, and add complexity only when it improves results. Anthropic's guide to building agents, which it has since updated, lists frameworks such as the Claude Agent SDK and the Strands Agents SDK from AWS, which handle common tasks like defining tools and chaining calls. Google Cloud offers the Agent Development Kit (ADK), an open-source Python SDK for building multi-agent systems.
Can AI agents talk to each other?
Yes. In a multi-agent system, several agents coordinate on a task, and Google Cloud says each one can use a different foundation model. IBM's surfing example has 1 agent consult another that specializes in surfing conditions.
Can I run an AI agent on my own computer?
Many agents run in the cloud, but the model at the center can also run locally if your hardware is up to it. Our guide to running AI locally covers what you need.
More in AI
- What is a large language model (LLM), and how does it work?
- What is generative AI, and how does it work?
- What is machine learning, and how does it work?
- What is prompt engineering, and how does it work?
- What is RAG in AI, and how does it work?