Running AI on an older laptop: what’s realistic

What’s realistic on a modest laptop

Think of a small local model as a private assistant for drafting, summarizing and answering questions, not a replacement for the largest online chatbots. Model makers design some models for exactly this kind of hardware. Google says its smaller Gemma 4 models are “specifically designed for efficient local execution on laptops and mobile devices,” and Microsoft lists “memory/compute constrained environments” as a main use for Phi-4-mini.

Speed depends heavily on your hardware, and Ollama’s own download page puts it plainly: “Large models are slow on a computer without a strong GPU.” On an older laptop, stick to small models.

What decides whether a model will run

Memory: RAM and VRAM

A model has to be loaded into memory before it can answer anything. If you have a supported graphics card, it goes into the card’s own memory (VRAM). Otherwise it goes into ordinary system RAM and runs on the processor. Tools such as llama.cpp can also split a model between the two, using the graphics card to speed up part of the work.

The longer the conversation or document you give a model, the more memory it needs on top of the model itself. Ollama’s documentation notes that required memory grows with the context length (how much text the model keeps in view).

LM Studio recommends at least 16 GB of RAM on Windows, and at least 4 GB of dedicated VRAM if you use a graphics card. It says 8 GB Macs may still work, but with smaller models and shorter contexts.

Model size

Models are described by their number of parameters, written as “1B” for 1 billion or “4B” for 4 billion. More parameters means more memory, as the table below shows. On an older laptop, models in the 1B to 4B range are the realistic choice.

Quantization, explained simply

Quantization is compression for AI models. Hugging Face describes it as storing a model’s weights “in a lower precision while trying to preserve as much accuracy as possible.” It’s a bit like rounding every number in a huge spreadsheet to fewer decimal places: the file gets much smaller, and the results stay close to the original.

Google’s published figures for Gemma 4 show how much this matters. These numbers cover loading the model only, not the extra memory for context or the app itself:

When you browse models in these apps, you’ll see labels such as “4-bit” or “Q4” for quantized versions. llama.cpp, which LM Studio uses as one of its runtimes, supports everything from 1.5-bit to 8-bit.

Processor and graphics

Check your processor before you install anything. LM Studio requires AVX2 support on standard (x64) Windows PCs. Intel’s spec page for each processor shows this under “Instruction Set Extensions”: the Core i5-7200U, for example, lists AVX2. GPT4All’s Windows and Linux builds ask for an Intel Core i3 2nd Gen, an AMD Bulldozer processor or better.

For graphics, Ollama supports NVIDIA cards with compute capability 5.0 or newer, some AMD Radeon cards, and Apple’s M-series chips. On an Intel Mac, Ollama runs on the processor only.

Beginner tools

If you just want to try it, start with LM Studio or GPT4All. Note that Ollama also offers cloud models, which run on Ollama’s servers rather than your laptop.

Small open models to try

These families each include a model small enough for a modest laptop. New versions come out often, so check each maker’s page for the latest.

• Gemma 4 E2B and E4B (Google, Apache 2.0 license): Google’s smallest current models, aimed at laptops and phones.
• Llama 3.2 1B and 3B (Meta, Llama 3.2 Community License): Meta says these are meant for “highly constrained environments, such as mobile devices.”
• Qwen3.5 0.8B and 2B (Qwen team, Apache 2.0 license): small members of the Qwen3.5 family.
• Phi-4-mini (Microsoft, MIT license): 3.8 billion parameters, built for memory- and compute-limited setups.

Why run AI locally: privacy

The main benefit is that your prompts stay on your computer. Ollama’s FAQ says, “We don’t see your prompts or data when you run locally.” LM Studio says that once a model is downloaded, “Nothing you enter into LM Studio when chatting with LLMs leaves your device,” and that it can operate entirely offline. You need the internet mainly to find and download models, runtimes and app updates.

The limits

• Capability: a model that fits in a few gigabytes holds far less than the huge models behind online chatbots. Double-check facts, figures and code.
• Speed: without a supported graphics card, everything runs on the processor. Expect to wait, especially with long documents.
• Storage: each model is a separate download. Small 4-bit models are a few gigabytes each (see the table above), and Ollama notes that larger models can take tens to hundreds of gigabytes.
• Memory: while a model is loaded, that memory isn’t available to other apps. Close your browser if things slow down, and see our guide to how much RAM you need.

If your laptop struggles even with the smallest models, memory is usually the limit, so more RAM is the upgrade to look at. If it can’t be upgraded, a refurbished laptop with 16 GB of RAM or an Apple silicon Mac is a realistic next step. Look up any model in our Can It Run? database first.

Frequently asked questions

Can I run an LLM locally without a GPU?

Yes. GPT4All says no GPU is required, and llama.cpp and Ollama both run models on the processor using system RAM. It works, but it’s slower than using a supported graphics card, so stick to small models.

How much RAM do I need to run an LLM locally?

LM Studio recommends 16 GB. With 8 GB you can still run the smallest models: Google lists 2.9 GB to load Gemma 4 E2B in 4-bit form, plus whatever the conversation and the app need on top.

Can I run an LLM locally without coding?

Yes. LM Studio and GPT4All are ordinary desktop apps: you install them, pick a model from a list, and chat in a window. Ollama is close: after installing it, one command (for example ollama run gemma4:e2b) downloads a model and starts a chat. llama.cpp is mainly for people comfortable with the command line.

Is running an LLM locally private?

When the model runs on your own computer, your prompts stay there. Ollama and LM Studio both say that what you type into a local model stays on your device. Cloud models offered inside the same apps are different: Ollama says it processes prompts for its cloud models to provide the service.

More in Low-Spec Performance

• How to speed up an old laptop: 10 checks, in order
• Games for integrated graphics: 13 that run without a graphics card
• Laptop overheating: what causes it and how to fix it
• Windows power mode: what it does and which to choose
• Is a graphics card upgrade worth it in an old desktop?

Sources

• LM Studio: System requirements
• LM Studio: Offline operation
• Ollama: Windows and macOS documentation
• Ollama: Hardware support (GPU)
• Ollama: FAQ
• Ollama: Download page
• Nomic: GPT4All on GitHub
• llama.cpp on GitHub
• Hugging Face: Quantization overview
• Google: Gemma 4 overview and memory requirements and model card
• Meta: Llama 3.2 model card
• Qwen: Qwen3.5-0.8B model card
• Microsoft: Phi-4-mini-instruct model card
• Intel: Core i5-7200U specifications