OpenAI embeddings, or text turned into numbers

OpenAI embeddings, or text turned into numbers

An embedding turns a fragment of text into a list of numbers chosen so that texts with similar meaning sit close together. Semantic search, recommendation systems, document clustering, and the whole context retrieval layer in language model applications rest on that.

OpenAI offers two models in this family, a smaller and a larger one, plus an older one the vendor advises against for new projects. That lineup has not changed for a long while, and that fact matters when choosing, as covered in its own section.

The practical advantage is mundane: it is the simplest route to working semantic search. One API call, no infrastructure, and costs measured in pennies at typical volumes.

Two models and choosing between them

The smaller model returns a vector of one thousand five hundred and thirty six dimensions and costs around two cents per million tokens. The larger returns three thousand and seventy two dimensions and costs thirteen cents per million tokens, six and a half times as much.

The selection rule is shorter than you might expect. Start with the smaller one. For most uses the accuracy difference is slight, while the difference in storage and search cost grows linearly with dimension count, so you pay it on every query rather than only at indexing time.

Reach for the larger one once you measure that the smaller is not enough. Not "when you suspect" but "when you measure against your own set of questions with expected answers", since that is the only way to find out.

The older model still works and costs five times more than the smaller new one at lower quality. If you have a project built on it, moving is worthwhile, though it requires recomputing the entire set, since vectors from different models are not comparable.

Your first call

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=["What is the notice period?", "Contract termination conditions"],
)

vectors = [d.embedding for d in response.data]

Note the list passed as input. One call accepts many texts at once, and that is the simplest optimisation in this whole area and equally the most often skipped. Processing a thousand fragments one at a time is a thousand round trips to the API, while batching by a hundred is ten. The difference in indexing time can be many times over.

Comparing two vectors is a dot product, since models in this family return normalised vectors.

import numpy as np

def similarity(a, b):
    return float(np.dot(a, b))

A result near one means texts of similar meaning, near zero means no relation. Worth knowing that values rarely drop below zero on natural text, so a cutoff threshold is chosen empirically rather than from theory. In practice that means a number like 0.7 says nothing on its own, and you need to see the score distribution for your own collection before setting any threshold.

Truncating dimensions

This feature is underrated and on larger collections decisive for costs.

Models in this family let you shorten the returned vector without recomputing it. You state a dimension count below the default and receive a shorter vector that still works sensibly, since the model was trained to pack the most important information at the start of the list.

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=chunks,
    dimensions=512,
)

The gain is twofold. The vector database takes three times less space, and search runs faster, since it compares shorter vectors. The accuracy loss is often small enough that in many uses it cannot be noticed without measuring.

The practical advice runs: test this on your own data before building the index. Take fifty real questions with expected answers, index the same collection at full and truncated dimension counts, and compare accuracy. If the difference is one percent and the saving three quarters of the space, the decision makes itself.

A technical caveat: truncated vectors deserve normalising, since cutting disturbs their length. Most libraries handle that themselves, while a custom implementation makes it easy to forget.

Costs and batch mode

The embedding bill is usually the smallest line in a language model application's budget, and saying that plainly matters, since people waste time optimising the wrong thing.

Indexing a hundred thousand fragments of five hundred tokens is fifty million tokens, roughly a dollar with the smaller model. Search costs even less, since a query is a dozen or so tokens.

The real costs sit elsewhere: in calls to the model generating answers and in maintaining the vector database. That is where savings are worth hunting.

If you are nevertheless indexing millions of documents, batch mode halves the price in exchange for results delivered within a day. For a one off archive run that is an obvious choice; for ongoing indexing it is useless.

Build a cache for vectors too. The same fragment of text processed twice yields the same result, so storing a content hash alongside the vector lets you skip everything unchanged. On an incrementally updated archive that turns an overnight full recompute into a few minutes of catching up.

Plan for the API side limits separately. Indexing a large collection will hit a cap on requests per minute, and while the library retries such responses by default, without your own exponential backoff the process can stall for hours. A sensible arrangement is batches of a few dozen fragments, a handful of parallel workers, and progress written to a file, so an interrupted run resumes where it stopped rather than from the beginning.

Chunking, the decision that outweighs the model

Embeddings are computed for fragments, and how you split text into fragments affects search accuracy more than the choice between two models does. It is the most often skipped part of the whole process.

A fragment too short carries too little context to answer anything. Two sentences torn from the middle of a contract sound sensible and say nothing about which party they concern or under what circumstances they apply.

A fragment too long dilutes the meaning. The vector averages the whole content, so a page of text spanning four different topics produces a vector sitting close to none of them.

A sensible starting point is fragments matching document sections, a few hundred to about fifteen hundred characters long, with a small overlap. The overlap solves the problem of a sentence cut in half by a fragment boundary.

Two things improve accuracy more than tuning length does. The first is prefixing each fragment with its section heading and the title of the document it came from. A fragment starting with "Chapter 4, Termination conditions" carries context the sentence alone lacks. The second is filtering out headers, footers, and page numbers before indexing, since repeating text present in every fragment dilutes its meaning.

Review twenty generated fragments before building anything further. A quarter of an hour of that work usually shows the parameters need adjusting, and that changes answer quality more than any prompt tuning.

Hybrid search

Embeddings alone carry one weakness worth knowing before you build a whole retrieval layer on them.

Meaning based search misses exact matches. A query for an invoice number, an error code, a proper name, or a product identifier may fail to reach the document containing exactly that value, since vectors operate on meaning and a string like a product code carries no meaning for the model.

The answer is combining two search methods: the classic one based on word matching, and the semantic one based on vectors. Results from both merge into one list, giving a system that copes with a descriptive question and a specific number alike.

Most vector databases support this directly today, so building it yourself is unnecessary. Do check that yours does before deciding, though, since adding it later means a second index alongside the first. If Elasticsearch already runs in your system you get this almost for free: an inverted index engine handles keyword matching and vector matching in a single query, and adding a separate vector store rarely pays off in that case.

A third element, usually added last, is reordering results with a separate model scoring the fit between a query and a fragment. That costs an extra call per query and can raise accuracy noticeably, since such a model sees the query and the fragment together rather than only the distance between two points in space.

The state of the competition

Here something the vendor's material will not say deserves stating, and it matters when choosing for a new project.

OpenAI's offering in this area has not changed for a long time, while during 2026 rivals released models going further. Options appeared handling several kinds of content at once, images and audio included, along with models specialised in particular domains, performing better on legal texts or on code.

That does not mean OpenAI's models stopped being a good choice. It means they stopped being the default choice made without thought. If you are building search over text documents in English or another major language, they remain sensible, cheap, and predictable. If you need image handling, code search, or high quality in a narrow domain, the alternatives deserve checking.

Two alternatives deserve checking first, because they sit at opposite ends of the scale. BGE is a family of open weight models you run yourself, paying only for hardware, which changes the arithmetic fundamentally at volume. Vertex AI embeddings go the other way, adding control over the processing region and billing inside Google Cloud.

The way to check is always the same and takes an afternoon. A set of fifty real questions with expected answers, the same collection indexed with two models, and a comparison of how often the right document landed in the top five results. Public benchmarks measure tasks that rarely resemble yours.

OpenAI embeddings against the alternatives

The last row deserves attention, since it gets skipped. Locally run embedding models are good enough today that on company document search the gap against paid services is often small, and the data never leaves your infrastructure. The cost is hardware and upkeep, so on a small collection it does not pay off while on a large one it can decide.

Common mistakes

The first is sending fragments one at a time instead of in batches. The difference in indexing time can be many times over, and the change is one line of code.

The second is mixing vectors from different models in one index. They are not comparable, so search returns random results. Changing model requires recomputing the whole collection.

The third is choosing the larger model without measuring. You pay six and a half times more and raise storage costs, often with no noticeable accuracy difference.

The fourth is skipping dimension truncation on a large collection. A three times smaller index at a loss of about a percent is usually a good trade.

The fifth is optimising embedding cost instead of answer generation cost. The first line is usually dozens of times smaller than the second.

The sixth is having no cache on an incrementally updated archive. Recomputing everything on every change is cost and time thrown away for no reason.

FAQ

Which model should I start with?

The smaller of the two current ones. For most uses the accuracy gap against the larger is slight, while storage and search costs grow with dimension count. Reach for the larger only once you measure that the smaller falls short.

What does it cost?

Indexing a hundred thousand fragments of five hundred tokens is around a dollar with the smaller model. Batch mode halves that in exchange for results within a day. This is usually the smallest line in a language model application's budget.

Can I shorten a vector after the fact?

Yes, models in this family accept a lower dimension count at call time without the result losing sense. On a large collection that means a three times smaller index at a small accuracy loss worth measuring on your own data.

Does changing model require recomputing everything?

Yes. Vectors from different models sit in different spaces and are not comparable, so mixing them in one index produces random results. Factor that into planning, since recomputing a large archive takes time.

Where should the resulting vectors live?

In a vector database or in an extension to a relational one. On a small collection the option covered in the piece on pgvector suffices; on a larger one Qdrant or Chroma deserve consideration.

The models are documented on OpenAI's site, and rates appear on the pricing page.