Llama, a model you can download and run yourself

Llama, a model you can download and run yourself

Llama is Meta's family of language models released with weights, meaning the model itself rather than only access through an API. The difference is fundamental: a closed model you rent, a weights model you download, run on your own hardware, and nobody knows what you ask it.

That matters in three situations: when data cannot leave the company, when per token cost at volume exceeds hardware cost, and when you need a model tuned to your own domain.

How it differs from closed models

The first difference concerns where the computation physically happens. A closed model processes your text on the vendor's server; a weights model runs on your machine or on a cloud provider of your choosing. For some industries that is the only acceptable arrangement.

The second is cost. A closed model bills per token, so the invoice grows linearly with traffic. Your own model bills per machine hour, so cost is fixed and comes out cheaper at volume and dearer at low volume, since the machine runs regardless.

The third is durability. A model downloaded once keeps working when a vendor changes pricing, retires a version, or shuts a service down. An application on a closed model must migrate at every such event.

The fourth is quality. Here the advantage sits with closed models, particularly on hard reasoning and long agentic tasks. The gap narrows with each generation but persists, and when choosing it is worth naming that plainly rather than hoping benchmarks settle it.

Fourth generation variants

The fourth generation brought two publicly released models built on a mixture of experts architecture, where only part of the parameters work on each token.

The lighter variant stands out for a very long context window, reaching millions of tokens, which lets you drop entire documentation or a sizeable repository into a request. The stronger variant is a general model for more demanding tasks.

A largest model, at close to two trillion total parameters, was also announced but has not been released with weights. Keep that in mind when planning, since an announcement is not availability.

The dates deserve knowing too. Both released models appeared in early April 2025, no newer generation has shipped with weights since, and Meta's leadership has publicly reserved the right not to put every one of its strongest models into open circulation. The family works and you can still download it, but when planning two years out, do not assume another generation at the same cadence.

The mixture of experts architecture carries one practical consequence easy to overlook. The parameter count active during generation is smaller than the total, so the model computes quickly, yet the whole thing must load into memory. You size hardware by the total figure and get the speed of a smaller model.

The licence, a thing to read

The models are not released under an open source licence in the classic sense but under a bespoke community licence. That distinction has legal consequences, so it is worth knowing before deployment.

The licence permits commercial use, which covers most applications. It does impose conditions: an acceptable use policy applies, attribution is required, and at very large operating scale further requirements appear.

The practical conclusion is that for a typical company the licence is no obstacle, yet it is also not the same as a permissive licence. If your legal team asks about terms, answering "it is an open model" is imprecise and naming the licence is better.

When you need a licence without conditions, consider models released under Apache 2.0, which have grown in number and in some applications match this family.

Running locally

The simplest route runs through a tool that downloads the model and starts a local server with one command.

ollama pull llama4:scout
ollama run llama4:scout

For application integration you use an interface compatible with a popular format, so code written against a cloud vendor works after changing the address.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="local")

response = client.chat.completions.create(
    model="llama4:scout",
    messages=[{"role": "user", "content": "Summarise the text below in three points: ..."}],
)

That compatibility matters more than it appears to. It lets you compare a local model against a cloud one on your own data without rewriting the application, and that is the only meaningful test when choosing.

More on the tool for running models locally sits in the piece on Ollama, which became the default way of working with this family on your own hardware.

Hardware requirements

This is the most common source of disappointment, so approach it numerically. Memory demand follows from parameter count and from the precision the model is stored at.

Context memory adds to that and grows with length. A very long context window looks attractive in a description and in practice needs memory an ordinary computer does not have.

Here comes a point easily missed while reading about the fourth generation. Its lighter model holds one hundred and nine billion parameters in total, so the four bit package in the Ollama registry weighs sixty seven gigabytes, and the vendor states plainly that the model fits only on a single server class card. On a desktop machine you will not run it at all.

A sensible starting point for home hardware is a computer with around thirty two gigabytes of unified memory or a card with twenty four gigabytes, and what fits there are the smaller models of the family's earlier generations, with the eight billion parameter one as the workhorse. The seventy billion variant needs roughly thirty five gigabytes at four bit quantisation, already more than a typical card offers.

If the hardware falls short, the alternative is renting a weights model from a cloud provider. You keep independence from a single company and the ability to fine tune, losing full data privacy.

Fine tuning to your domain

The biggest advantage of a weights model surfaces on specific tasks, where a general model performs moderately because it knows neither your vocabulary nor your conventions.

Low rank fine tuning changes only a small set of additional parameters, so it fits on a single card and takes hours rather than weeks. The result is a small file layered onto the base model.

Three things decide success. The first is data: several hundred good examples beat ten thousand random ones. The second is a validation set held out before you start, since without it you cannot tell whether the model improved or merely learned the training set. The third is comparison against something simpler: a well written instruction with a few examples in the prompt often gets close with no training at all.

Start with that last one. Fine tuning makes sense once the prompt has hit its ceiling, not instead of working on the prompt.

Production deployment, or what lies beyond a laptop

A tool for running a model locally suits one person working. An application serving many users needs a server handling many requests at once, and that is a different class of software.

Servers built for production deployments batch requests and share the attention cache between them, so throughput rises severalfold against one at a time handling. At ten concurrent conversations the difference is a multiple rather than a percentage.

The second thing is limits. A model with no cap on response length and concurrent requests will fill memory and start refusing connections. Set those values from measurement rather than assumption.

The third is monitoring. Three numbers matter: time to first token, tokens per second, and card memory usage. The first speaks to how it feels to a user, the second to throughput, the third to how close you are to the wall.

The fourth is a plan for growth. A model fitting one card at five users needs either a second machine or a smaller variant at fifty. The latter direction is often better, since a smaller model with a good prompt often suffices and costs an order of magnitude less.

Context and retrieval instead of fine tuning

A very long context window tempts you to drop entire documentation into a request and skip building retrieval. Sometimes that is right; more often it is the most expensive way to answer a question.

For one off analysis long context wins: reading a lengthy document, comparing two specification revisions, going through a log dump. You do it once, and the model sees the whole thing and catches contradictions between distant passages.

In an application answering user questions it loses. A hundred thousand context tokens per question means processing time and occupied memory, so a vector database such as Chroma or the pgvector extension comes out many times cheaper and faster.

The practical rule: long context for one off, bulky tasks and retrieval for repeated ones. Mixing the two, retrieval plus moderate context, works best in most deployments.

Llama against the alternatives

The choice is rarely exclusive. A common arrangement runs a local model for classification and bulk processing where cost decides, and a closed model for hard tasks where quality does. A thin layer with one interface routes traffic to either.

That split has one more advantage, rarely discussed. Routing part of the traffic to your own model gives a real reference point for how good that option is on your data. Without that comparison, a decision to move across entirely rests on somebody else's benchmarks, and those measure something other than your case.

Note too that the mere ability to leave a closed model is often worth more than leaving. A vendor whose only possible customer you are negotiates differently from a vendor who can be replaced within a week.

Common mistakes

The first is comparing models on somebody else's benchmarks. Public results measure tasks that rarely resemble yours, and differences between models depend heavily on language and domain.

The second is overlooking languages other than English. Open models perform worse outside English, so a test on your own data is mandatory here rather than advisable.

The third is picking a variant by parameter count without checking memory. A model that does not fit in card memory falls back to the processor and runs many times slower.

The fourth is calling these models open source in company documents. The licence is bespoke and conditional, so precision here saves conversations with the legal team.

The fifth is fine tuning instead of improving the prompt. It is a costlier and slower route to a goal reached in half the cases by an instruction and a few examples.

The sixth is forgetting fixed costs. A machine with a graphics card costs the same on a quiet day as at peak, so at modest volume a per token model often comes out cheaper.

FAQ

Is Llama free?

The weights are available at no charge, but the licence is not a classic open source licence. Commercial use is permitted subject to conditions, including an acceptable use policy and an attribution requirement. At very large operating scale further clauses apply, so read the licence text.

What hardware is needed?

Smaller models from earlier generations run at four bit quantisation on a computer with around thirty two gigabytes of unified memory or a card with twenty four gigabytes. Both fourth generation models already need a server class machine, since even the lighter one takes tens of gigabytes in weights alone, and renting from a cloud provider is then often the simplest route.

Does Llama match closed models?

On typical tasks such as summarisation, classification, and simple generation the gap is slight. On hard reasoning, code work, and long agentic tasks the leading closed models still perform better. Run the comparison on your own data, since differences depend on domain.

How does Llama handle languages other than English?

Worse than English, which holds for most open models. For applications in another language, test several variants on your own examples, since differences there can exceed those between generations of the same model.

When does a local model come out cheaper?

When volume is high and steady. A machine costs the same regardless of traffic, so the break even point depends on how many calls a day you serve. At a few hundred calls a day a per token model usually wins; at tens of thousands the advantage moves to your own hardware.

The models and licence are described on the project site, the weights come through Hugging Face, and the model cards and helper utilities sit in Meta's GitHub repository.