What is a large language model (LLM), and how does it work?
What is a large language model? It's a type of AI model trained on enormous amounts of text so it can understand and generate language. An LLM works by predicting the next small piece of text, called a token, over and over until it has written a full answer. LLMs power chatbots such as ChatGPT, Claude, Copilot and Gemini.
The short version
- A large language model (LLM) is a deep learning model trained on huge amounts of text, usually built on an architecture called the transformer.
- It writes by predicting 1 token at a time. Google's course calls LLMs "essentially autocomplete mechanisms" that can predict thousands of tokens.
- "Large" refers to the number of parameters. Google says transformers contain hundreds of billions or even trillions of them.
- LLMs are trained first on general text, then often fine-tuned to follow instructions and give safer, more useful answers.
- Google says LLMs hallucinate, meaning their predictions "often contain mistakes."
What is a large language model in simple terms?
IBM defines large language models as "a category of deep learning models trained on immense amounts of data, making them capable of understanding and generating natural language and other types of content to perform a wide range of tasks." Google Cloud calls an LLM "a statistical language model" that can generate and translate text and perform other natural language processing tasks.
Microsoft's training course suggests thinking of an LLM as a super-powerful version of the predictive text feature on many phones. You give it a prompt, and it predicts the most probable continuation, 1 token at a time.
The difference from phone autocomplete is scale. IBM says LLM training starts with billions or trillions of words, and the finished model has learned patterns in grammar, facts, reasoning structures and writing styles. AWS says 1 model can perform completely different tasks, such as answering questions, summarizing documents, translating languages and completing sentences.
How large language models work
Tokens: how an LLM reads text
An LLM splits text into tokens, which Google describes as a word, part of a word or even a single character. Microsoft's example is the "un" in "unbelievable" and "unlikely," which can be its own token.
Google's machine learning course says that for English, 1 token is about 4 characters or 3/4 of a word, so 400 tokens is roughly 300 English words. The ratio differs between languages.
Embeddings: turning tokens into numbers
Each token is mapped to a list of numbers called an embedding. Words used in similar contexts end up with similar embeddings. In Microsoft's example, the embeddings for "dog" and "puppy" point in roughly the same direction, while "skateboard" points somewhere very different.
The transformer and self-attention
Google calls the transformer the most successful and widely used architecture for building LLMs. IBM dates it to the 2017 research paper "Attention Is All You Need." Its key idea is self-attention: for each token, the model asks how much every other token in the input affects its meaning.
Google uses this sentence to show why that matters: "The animal didn't cross the street because it was too tired." Self-attention helps the model work out that "it" refers to the animal. Change the last word to "wide" and self-attention would ideally rate "street" as more relevant instead.
IBM adds that transformers can process text in parallel, which made training on unprecedentedly large datasets practical.
Generating an answer
When you send a prompt, the model calculates probabilities for every possible next token, picks one, adds it to the text and repeats. IBM calls this process inference and stresses that "the model does not 'know' the final answer in advance." It makes its best guess at every step.
The model doesn't always take the single most likely token. Google's course says an application might pick the highest-probability option or a random one from the tokens above a probability threshold. IBM says a setting called temperature controls how random the generated text is.
How LLMs are trained
Building an LLM happens in stages:
- Pre-training. IBM says the model learns from a massive body of text, including books, articles, websites and code. It says this stage uses self-supervised learning, which doesn't require labeled datasets. Google's course describes training on masked predictions: some tokens are hidden and the model learns by trying to predict them. Its example hides words in "The residents of the sleepy town weren't prepared for what came next."
- Fine-tuning. A pre-trained model is then trained further on a much smaller, task-specific dataset. IBM's example is fine-tuning a general model on legal questions and answers to make a chatbot for law.
- Instruction tuning. Google and IBM both describe this as training that improves a model's ability to follow instructions. IBM explains that pre-trained LLMs are not inherently optimized for following instructions or conversational goals.
- Reinforcement learning from human feedback (RLHF). People rank the model's answers and the model learns to prefer the higher-ranked ones. IBM says this is often used to make outputs useful, safe and consistent with human values.
Google's course warns that you will probably never train an LLM from scratch. It takes an enormous training set, months of computing time and huge amounts of electricity. IBM says the data, GPUs, energy and expertise involved are why most popular LLMs are built by large tech companies.
Why is it called a "large" language model?
"Large" refers to the model's size, measured in parameters: the internal values, such as weights, that the model adjusts during training. Google's course says transformers contain "hundreds of billion or even trillions of parameters" and that research shows transformers with more parameters consistently outperform those with fewer.
For scale, IBM notes that OpenAI's GPT-3 (2020) had 175 billion parameters. IBM also says so-called small language models have comparatively few parameters, which suits them to smaller devices and resource-constrained environments.
LLM vs AI vs generative AI
These terms overlap but aren't interchangeable:
| Term | What the sources say | How it relates to LLMs |
|---|---|---|
| Artificial intelligence (AI) | The wider field; IBM says "all machine learning is AI, but not all AI is machine learning" | LLMs are a kind of AI |
| Machine learning | IBM: the subset of AI focused on algorithms that learn patterns from training data | IBM says machine learning is the backbone of LLMs |
| Deep learning | IBM: the subset of machine learning driven by large, "deep" neural networks | IBM classes LLMs as deep learning models |
| Generative AI | Google: a class of models that creates content such as text, images, audio and video from user input | Microsoft places LLMs at the core of generative AI |
| Small language model (SLM) | IBM: smaller in scale and scope, with comparatively few parameters | A lighter relative of the LLM |
Our guides to machine learning and generative AI explain those layers in more detail.
Some LLMs handle more than text. Google Cloud says LLMs can also be trained on code, images, audio and video. IBM describes ChatGPT, Claude, Copilot and Gemini as interfaces through which the public uses LLMs. The models behind those apps keep changing; Anthropic released Claude Fable 5.2 on November 9, 2026, and Google has announced Gemini 4 Carbon, which is going to cyber defenders first.
What large language models are used for
IBM and Google Cloud list common uses:
- Writing: drafting emails, blog posts and other text from a prompt.
- Summarizing: condensing long articles, reports and documents.
- Chatbots and assistants: answering questions in a conversation.
- Coding: writing code, finding errors and translating between programming languages.
- Translation: converting text between languages.
- Sentiment analysis: judging the tone of customer feedback at scale.
An LLM on its own only generates text. IBM says that when it's connected to memory, APIs and other systems, it can carry out tasks such as booking a flight. IBM calls this an agentic system; our guide to AI agents explains more. An LLM can also answer from your own documents through retrieval-augmented generation (RAG), which passes the relevant text into the model's context window.
Limits and risks of LLMs
- Hallucinations. IBM describes these as the model generating information that is false or misleading while sounding plausible. Check anything important against a reliable source.
- Bias. IBM says LLMs can reflect and amplify biases in their training data, producing unfair or offensive outputs.
- Cost and energy. Google and IBM both note that training and running LLMs take large amounts of computing power and electricity.
- A limited context window. The context window is the maximum number of tokens a model can use at once. IBM says newer LLMs can handle hundreds of thousands of tokens, but a model can't use text beyond its limit.
Frequently asked questions
What is an example of a large language model?
IBM says LLMs are easily accessible to the public through interfaces like Anthropic's Claude, OpenAI's ChatGPT, Microsoft's Copilot, Meta's Llama models and Google's Gemini assistant. Older examples include Google's BERT and OpenAI's GPT-3.
Is ChatGPT a large language model?
Yes, in AWS's description: it names ChatGPT alongside Claude 2 and Llama 2 as large language models. IBM describes ChatGPT more precisely as an interface through which the public uses LLMs.
Do large language models make mistakes?
Yes. Google's course says LLMs hallucinate, meaning their predictions often contain mistakes. IBM describes hallucinations as false or misleading information that sounds plausible, so check important answers against a reliable source.
Can I run a large language model on my own computer?
Yes, with the right model and hardware. IBM notes that many open-source models can be deployed locally, and that small language models suit smaller devices. Our guide to running AI locally explains what hardware you need.
When did large language models start?
IBM traces the modern LLM era to the transformer, introduced in 2017. Google's BERT followed in 2018, then OpenAI's GPT-2 in 2019 and GPT-3 in 2020.
More in AI
- What is an AI agent, and how does it work?
- What is generative AI, and how does it work?
- What is machine learning, and how does it work?
- What is prompt engineering, and how does it work?
- What is RAG in AI, and how does it work?