How to run Llama locally

Why run Llama on your own computer

Llama's weights are published by Meta, so unlike closed chatbots you can download a Llama model and run it on your own machine. Once it is downloaded, it works offline, your prompts never leave the computer, and there are no message limits or per-token bills. The trade-off is hardware: bigger models need a lot of memory, and a laptop will answer more slowly than a cloud server.

This guide covers the three most common tools, from easiest to most hands-on: Ollama, LM Studio and llama.cpp. If you only want to try Llama, our free Llama chat needs no installation at all.

Pick a model that fits your machine

The single most important choice is model size. A rough rule: you need at least as much free memory (graphics card memory, or unified memory on Apple silicon) as the model file, plus extra room for the conversation. If the model does not fit, it either refuses to load or runs very slowly from regular system memory.

These are the download sizes Ollama lists for its default versions of each Llama model. They are compressed (quantized) files, which is how most people run models at home.

If you are not sure, start with Llama 3.1 8B. It is small enough for many recent laptops and capable enough to be useful. Meta's own card for Llama 4 Scout notes that it fits on a single H100 data-center GPU with Int4 quantization, which gives a sense of the scale of the Llama 4 models.

Option 1: Ollama (easiest)

Ollama is an app that downloads and runs models with one command and keeps them ready for later use.

1. Install Ollama from its official site.
2. Open a terminal and run ollama run llama3.1. The first run downloads the model; after that it starts in seconds.
3. Type your question at the prompt and press Enter.

Other Llama models work the same way, for example ollama run llama3.2 or ollama run llama4:scout. The Ollama library pages for Llama 3.1, Llama 3.2 and Llama 4 list every available size. On the Llama 4 page, Ollama shows Scout with a 10M context window and Maverick with 1M, both accepting text and images.

Ollama also runs a local server, so your own scripts can use the model. A chat request looks like this:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "messages": [{"role": "user", "content": "Explain recursion in two sentences."}]
}'

Option 2: LM Studio (a desktop app)

LM Studio is a free desktop app with a chat window, so you never need a terminal.

1. Download LM Studio for your system and open it.
2. Open the Discover tab, search for "Llama" and download a version that fits your memory.
3. Go to the Chat tab, pick the model in the model loader at the top, and start chatting.

LM Studio can also act as a local server that speaks the same format as OpenAI's API, by default at http://localhost:1234/v1. Many existing tools can point at that address instead of a cloud service. See the LM Studio getting-started guide and the OpenAI-compatible server docs.

Option 3: llama.cpp (most control)

llama.cpp is the open-source engine that powers many local AI tools. Using it directly gives you the most control over performance settings. It runs models in the GGUF file format, supports quantization from 1.5-bit to 8-bit to shrink models, and treats Apple silicon as a first-class platform.

1. Install it. The project README gives one-line installers: curl -LsSf https://llama.app/install.sh | sh on macOS and Linux, or irm https://llama.app/install.ps1 | iex in Windows PowerShell.
2. Chat in the terminal with a GGUF model from Hugging Face: llama cli -hf <user>/<repo>.
3. Or start a local OpenAI-compatible server: llama serve -hf <user>/<repo>.

Replace the placeholder with the Hugging Face repository of a GGUF version of the Llama model you want. Read the repository's description to check which quantization each file uses; smaller files fit in less memory but lose some quality.

Option 4: the original weights from Meta

Developers who want the full-precision weights can download them from Meta's pages on Hugging Face, such as Llama 3.1 8B Instruct or Llama 4 Scout. These repositories are gated: you must sign in and accept the license on the model page before the files unlock. The model cards include example code for the Hugging Face Transformers library. Full-precision weights need much more memory than the quantized versions above, so this route is mostly for servers and fine-tuning.

Tips for a smooth start

• Start small. Try an 8B or 3B model first. Move up only if answers are not good enough and your hardware can cope.
• Close other heavy apps. Browsers with many tabs and games compete for the same memory.
• Keep context modest. Very long conversations use more memory and slow things down. Start a new chat for a new topic.
• Use a system prompt. Telling the model its role and preferred answer style helps small models a lot. Our prompt library has ideas.
• Remember the license. Running a model for yourself is simple, but if you build a product with it, read our Llama license guide.
• Expect older knowledge. Local models do not search the web. Llama 3.x knowledge ends in December 2023 and Llama 4 in August 2024.

Local or online: which to choose

Run Llama locally when privacy, offline use or heavy daily use matter more than convenience, and when you have suitable hardware. Use an online chat when you want the larger Llama 4 models without a powerful computer, or when you are simply trying Llama out. You can also call Llama from code through cloud providers; our Llama API guide covers that.

Try Llama 4 Maverick online first

FAQ

Can I run Llama without a graphics card?

Yes, small models such as Llama 3.2 3B and Llama 3.1 8B run on a regular processor, just more slowly. Larger models are impractical without a strong GPU or lots of unified memory.

Is running Llama locally free?

The software above and the Llama weights are free to download. Your only costs are the hardware and electricity.

Can a local Llama model read images?

Llama 4 Scout and Maverick accept images, and so do the Llama 3.2 Vision models. The 1B, 3B and 8B text models do not.

Which tool should a beginner use?

LM Studio if you prefer clicking through a desktop app, Ollama if you are comfortable typing one command in a terminal.