Every message you send to an online chatbot goes to a company’s servers. A local model runs entirely on your own computer instead: it works without an internet connection, costs nothing per message, and your conversations never leave the machine. That makes it useful for private documents, travel, or just curiosity about how these things work.
The trade-off is that the models small enough for a laptop are less capable than the big online ones. For writing help, summaries, brainstorming and everyday questions, they’re good enough to be genuinely useful.
What your laptop needs
The single most important thing is memory (RAM), because the whole model has to fit in it while it runs. If you’re buying a new laptop with this in mind, our student laptop guide explains why 16GB is the sensible minimum.
| Your memory | What runs well | Good for |
|---|---|---|
| 8GB | Small models, around 1–4 billion parameters | Short answers, simple rewriting; expect it to feel slow |
| 16GB | Mid-size models, around 7–14 billion parameters, plus OpenAI’s gpt-oss-20b | The sweet spot: solid everyday chat and writing help |
| 32GB or more | Larger models, around 27–32 billion parameters | Noticeably smarter answers, longer documents |
A few more things help:
- Apple Silicon Macs (M1 and later) are especially good at this, because the processor and graphics share the same fast memory.
- Windows laptops with an Nvidia graphics card run models much faster if the model fits in the card’s own memory (VRAM).
- Without a graphics card it still works on the processor alone, just more slowly.
- Storage: models are large files, typically 2–20GB each, so leave room.
LM Studio itself recommends at least 16GB of RAM.
Step 1: Install a free app
Two free apps make this easy. Both use the same engine underneath and download models for you.
- LM Studio (Windows, Mac, Linux): the easiest start. It has a chat window like ChatGPT, a built-in model browser, and it warns you when a model is likely too big for your machine.
- Ollama (Windows, Mac, Linux): lighter and runs quietly in the background. You download and run models with one command, such as
ollama run gemma3, and it now has a simple chat window too. Other apps can connect to it.
If you’ve never done this before, start with LM Studio.
Step 2: Download a model
Open LM Studio’s Discover (search) tab and pick a model from one of the main families:
- Google’s Gemma and Alibaba’s Qwen are strong all-rounders with small versions that run well on 8–16GB.
- OpenAI’s gpt-oss-20b is designed to run in about 16GB of memory and is good at step-by-step reasoning.
- Meta’s Llama and Mistral models are widely supported and come in many sizes.
Choose the newest version of a family, in a size your memory can handle. Versions marked Q4 (or “4-bit”) are compressed to use less memory with little loss in quality; they’re the usual choice on laptops.
Step 3: Start chatting
Load the model, wait a few seconds while it moves into memory, and type. The first reply is usually the slowest. On a 16GB laptop, a mid-size model typically writes at roughly reading speed.
Try things like:
- “Summarise this text in five bullet points”, then paste in a long email or article.
- “Rewrite this paragraph to sound more formal.”
- “Explain how a mortgage works, as if I’m 15.”
- “Give me ten name ideas for a cooking channel.”
Turn off Wi-Fi and try again: it still works.
What to expect
- It doesn’t know recent events. Each model only knows what was in its training data, up to a cut-off date, and can’t look things up unless you connect a search tool.
- It makes mistakes. Small models get facts wrong more often than large online ones. Check anything important.
- Speed depends on your hardware. If replies crawl, choose a smaller model or a more compressed version.
- Long documents are hard. Very long inputs use a lot of extra memory. Paste in sections rather than whole books.
Wondering how this compares with the assistants on your phone and speaker? See how voice assistants like Alexa and Siri actually work.
Going further
Once you’re comfortable, both apps can let you chat with your own files, so you can ask questions about a folder of PDFs or notes without uploading them anywhere. They can also act as a local server for other apps, including coding assistants and writing tools, so the AI features in those apps run on your machine instead of in the cloud.
FAQ
Is it legal to download these models?
Yes. Open-weight models such as Gemma, Qwen, Llama, Mistral and gpt-oss are released for anyone to download and run. Each has a licence; most allow personal and commercial use, with some conditions, so check the licence if you plan to build a product on one.
Will it damage my laptop?
No, but it works the processor and memory hard, so the fans spin up and the battery drains faster. Plug in for long sessions.
Is a local model as good as ChatGPT, Claude or Gemini?
Not the small ones that fit on a laptop. They're good at everyday writing, summarising and questions, but weaker at complex reasoning, long documents and up-to-date facts, and they can't search the web unless you set that up.



