Hugging Face, where open models live

Hugging Face, where open models live

Hugging Face is three things at once: a repository holding hundreds of thousands of models and datasets, a set of libraries for working with them, and a service that runs models without your own hardware. For most teams the first encounter looks the same: they search for a model for a specific task and land on the Hub.

What it consists of

The Hub stores models, datasets, and demos. It runs on git, so a model is a repository with change history, a card describing its intended use, and weight files. You can clone it, inspect it, and run it locally.

The libraries are the code layer. Transformers loads language and vision models in one call, Datasets fetches and processes datasets, Sentence Transformers computes embeddings, and Diffusers handles image generation.

Inference Providers runs models through an API with nothing to provision. A request routes to one of the compute partners, and billing follows that provider's rates with no Hugging Face markup.

Spaces hosts demos. An application in Gradio or Streamlit gets a URL and runs in the browser, which is the fastest way to show a model to somebody who will not open a notebook.

The Hub and the model card

Hub search filters by task, language, licence, and size. That last criterion gets skipped often, and it decides whether the model runs on the hardware you have at all.

The model card is the most important document and deserves reading before you download weights. It carries intended use, training data, known limitations, and the licence. A model excelling at English can stumble on another language, and one trained on texts from a single domain generalises poorly to another.

Check the weight format separately. Prefer safetensors, because the older format built on Python serialisation can execute code while loading a file. That is a real attack path rather than a theoretical one, so pulling weights from an unknown account without checking the format is a risk.

Model naming carries information too, though it is not standardised. A segment such as base or large, or a parameter count, indicates size, an instruct suffix marks a variant tuned to follow instructions, and chat one tuned for conversation. A model without those markers is usually a base model that continues text, and used as an assistant it behaves unexpectedly. This is a common reason for the impression that a model is weak when the wrong variant was simply chosen.

It also pays to watch download counts and the last update date. A model untouched for two years usually has a newer counterpart, and the quality gap can be wide.

Transformers in practice

Simple tasks need only a pipeline, which picks the tokeniser and configuration for you.

pip install transformers torch

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="cardiffnlp/twitter-xlm-roberta-base-sentiment"
)

print(classifier("The order arrived two days late, but the product is good"))

On first run the library downloads weights and stores them in a cache, by default in the home directory. On a machine building container images it pays to set that path deliberately, otherwise every build downloads the model again.

For embeddings you reach for Sentence Transformers, a library sitting on the same layer but reducing embedding to a single call instead of tokenising by hand and pooling the model output yourself.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-base")
vectors = model.encode(["first chunk", "second chunk"])

A multilingual model in the base size gives good results across languages and fits on an average graphics card, or even a CPU at modest volume. You then store the vectors in a database such as Pinecone or pgvector.

Inference Providers without your own GPU

If you would rather not maintain hardware, you call the model through an API. The request routes to one of the partners and you pay their rate.

from huggingface_hub import InferenceClient

client = InferenceClient(api_key=os.environ["HF_TOKEN"])

response = client.chat_completion(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Summarise the text below in three sentences: ..."}]
)

The interface follows the convention known from OpenAI, so swapping providers in existing code usually means changing a base URL and a key. That makes comparing an open model against a closed one on your own data straightforward, before you commit either way.

Every account receives a monthly credit allowance for experimentation. The Pro plan raises it substantially, and organisations on team plans share a pool among members and can bill all requests centrally even though each person uses their own token.

Spaces, a demo in minutes

A Space is a repository holding an application, most often in Gradio. You push code, the platform builds an image and exposes a URL.

import gradio as gr
from transformers import pipeline

summarise = pipeline("summarization", model="facebook/bart-large-cnn")

gr.Interface(
    fn=lambda text: summarise(text, max_length=120)[0]["summary_text"],
    inputs=gr.Textbox(lines=10),
    outputs="text"
).launch()

The free tier runs on CPU, which suffices for light models. Heavier ones need a stronger machine billed by the hour: from three cents an hour for a plain CPU upgrade, through single graphics cards, up to more than twenty three dollars an hour for an eight card rig. Pro accounts also receive a multiplied ZeroGPU allowance, where a card is assigned dynamically only for the duration of compute.

Spaces also earn their keep inside a team, not only for outside demos. An interface where somebody from customer support can paste ten real tickets and watch the model classify them yields better feedback than a table of metrics. Along the way you collect edge cases the engineering team would never have imagined.

A practical note: a Space exposed publicly with no limits attracts bots. If the demo costs you an hour of card time, add rate limiting or a simple access gate.

Pricing

The inference bill is separate from the subscription and depends on the model and the provider handling the request. At steady volume, price a thousand calls in practice, since differences between models of similar quality can run several times over.

Fine tuning on your own data

Fine tuning makes sense less often than people assume, yet in specific cases it delivers what no prompt can. Those are tasks where holding a fixed format matters, or narrow domain vocabulary, or a response style particular to an organisation.

Check three things before starting. Whether you have at least a few hundred good quality examples, since below that threshold the result can be worse than before. Whether the problem disappears once you add examples directly to the prompt, which is cheaper and reversible. Whether the task is stable over time, because every requirement change means tuning again.

For modest datasets the standard approach uses adapters, training a small addition to the weights rather than the whole model. The output takes tens of megabytes instead of tens of gigabytes, and training fits on a single graphics card.

from peft import LoraConfig, get_peft_model

config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(base_model, config)

Prepare training data as carefully as a test set. Split it before training: part for learning, part held out for evaluation and never touched. Evaluating on the same examples the model learned from always looks excellent and says nothing.

One side effect deserves attention: forgetting. A model tuned for a narrow task usually loses some general ability, so if it also has to handle conversation outside the domain, verify that separately before deployment.

Open model or vendor API

A rule that holds in most projects: start with a closed model to learn whether the task is solvable at all. Only once it works and you know the volume, calculate whether an open model lowers the bill enough to cover its upkeep.

Count the upkeep honestly while you are at it. A graphics card rented in the cloud full time costs several hundred dollars a month regardless of how many requests it serves, so the break even point against an API sits higher than the per token rate suggests. An open model starts winning under steady heavy load, not under traffic concentrated into a few hours a day.

For simple high volume work such as classification or embeddings, the open model wins almost every time. For tasks demanding reasoning, the quality gap can still show.

Licences you have to watch

Open weights do not mean unrestricted use. Models on the Hub carry different licences, and some limit commercial use, require attribution, or forbid using outputs to train competing models.

Three things deserve checking before deployment. The model licence itself, since it can differ from the code licence in the same repository. The licence of the dataset the model was trained on, because restrictions can carry into the result. Any terms acceptance requirement, visible as a form before the weights download, which in an automated pipeline surfaces as an authentication error.

The provenance of training data is a separate question the licence does not always answer. Image and code generating models attract disputes about what they were trained on, and the risk passes to whoever publishes the output. Where the result reaches a customer as a product, settle that with a lawyer rather than from a line in a model card.

Inside a company it makes sense to keep a short list of approved licences and check it when adding a model to a project. Discovering a year later that a production model carries a non commercial licence is a legal problem, not a technical one.

Common mistakes

The first is picking the largest model that fits. The base size often suffices for classification or embeddings, while running several times faster and cheaper.

The second is downloading weights on every container start. Set a cache directory and mount it as a volume, otherwise service startup takes minutes and generates pointless traffic.

The third is ignoring the tokeniser when comparing. The same text consumes a different number of tokens across models, so a cost comparison without that is misleading.

The fourth is using a model without reading its card. A model trained on data from one domain can behave unpredictably in another, and the card usually says so.

The fifth is keeping an access token in code. A token with write permission can modify organisation repositories, so treat it as a production key.

FAQ

Is Hugging Face free?

A free account gives access to public models and datasets plus a small inference credit allowance. Pro costs 9 USD per month and raises limits, Team 20 USD per user, Enterprise 50 USD per user.

Storage for private repositories bills separately, which is easy to forget on a platform associated with hosting models for free. The rate starts at eighteen dollars per terabyte per month and falls at larger volumes. Egress and the content delivery network are included, so the bill depends on how much you keep rather than how much you download. For a team versioning several variants of a large model, that line grows faster than the subscription. Spaces on a stronger machine bill separately, from three cents an hour for an upgraded CPU upwards.

Can I use the models commercially?

It depends on each model's licence, since they differ. Some allow any use, some restrict commercial use or attach extra conditions. Check the licence of both the model and its training dataset before deploying.

How does Hugging Face differ from Ollama?

Hugging Face is a repository plus services around models, Ollama is a tool for running models locally with one command. They do not compete directly: many models available in Ollama come from the Hub in the first place.

Do I need a graphics card?

For embedding models and smaller classifiers a CPU suffices, especially at modest volume. Language models in the billions of parameters need a card, or Inference Providers where the hardware sits on the vendor side.

How do I start without training my own model?

Find an existing model for your task, run it through a Transformers pipeline or the API, and check quality on your own data. Training your own makes sense only once no existing model performs well enough and you have data to fine tune with.

Billing rules are described in the Inference Providers documentation, and the models sit on huggingface.co.