cruntrix
AIHOW-TO

How to run an AI model on your own laptop

A private chatbot that works offline, costs nothing per message and never sends your words to a server. What your laptop needs, which free app to use, and what to expect.

Illustration of a computer chip labelled AI

Every message you send to an online chatbot goes to a company’s servers. A local model runs entirely on your own computer instead: it works without an internet connection, costs nothing per message, and your conversations never leave the machine. That makes it useful for private documents, travel, or just curiosity about how these things work.

The trade-off is that the models small enough for a laptop are less capable than the big online ones. For writing help, summaries, brainstorming and everyday questions, they’re good enough to be genuinely useful.

What your laptop needs

The single most important thing is memory (RAM), because the whole model has to fit in it while it runs. If you’re buying a new laptop with this in mind, our student laptop guide explains why 16GB is the sensible minimum.

Your memory What runs well Good for
8GB Small models, around 1–4 billion parameters Short answers, simple rewriting; expect it to feel slow
16GB Mid-size models, around 7–14 billion parameters, plus OpenAI’s gpt-oss-20b The sweet spot: solid everyday chat and writing help
32GB or more Larger models, around 27–32 billion parameters Noticeably smarter answers, longer documents

A few more things help:

LM Studio itself recommends at least 16GB of RAM.

Step 1: Install a free app

Two free apps make this easy. Both use the same engine underneath and download models for you.

If you’ve never done this before, start with LM Studio.

Step 2: Download a model

Open LM Studio’s Discover (search) tab and pick a model from one of the main families:

Choose the newest version of a family, in a size your memory can handle. Versions marked Q4 (or “4-bit”) are compressed to use less memory with little loss in quality; they’re the usual choice on laptops.

Step 3: Start chatting

Load the model, wait a few seconds while it moves into memory, and type. The first reply is usually the slowest. On a 16GB laptop, a mid-size model typically writes at roughly reading speed.

Try things like:

Turn off Wi-Fi and try again: it still works.

What to expect

Wondering how this compares with the assistants on your phone and speaker? See how voice assistants like Alexa and Siri actually work.

Going further

Once you’re comfortable, both apps can let you chat with your own files, so you can ask questions about a folder of PDFs or notes without uploading them anywhere. They can also act as a local server for other apps, including coding assistants and writing tools, so the AI features in those apps run on your machine instead of in the cloud.

FAQ

Is it legal to download these models?

Yes. Open-weight models such as Gemma, Qwen, Llama, Mistral and gpt-oss are released for anyone to download and run. Each has a licence; most allow personal and commercial use, with some conditions, so check the licence if you plan to build a product on one.

Will it damage my laptop?

No, but it works the processor and memory hard, so the fans spin up and the battery drains faster. Plug in for long sessions.

Is a local model as good as ChatGPT, Claude or Gemini?

Not the small ones that fit on a laptop. They're good at everyday writing, summarising and questions, but weaker at complex reasoning, long documents and up-to-date facts, and they can't search the web unless you set that up.

SOURCES

AI DeskCruntrix's AI coverage: tools compared, how-tos and new releases explained, researched from published sources and edited by Nate Arden.