Google embeddings, or five kinds of content in one space

Google embeddings, or five kinds of content in one space

Google offers embedding models through two routes: a simpler API for developers and a cloud platform for enterprise deployments. The models are the same; what differs is the access layer, billing, and the management machinery you get around them.

The newest model's differentiator is the largest in this whole category. Beyond text and images it accepts video, audio, and whole documents, embedding everything in one shared space. A query written in words can therefore find a passage in a recording or a scene from a film, without describing them in text.

This text covers the embedding models themselves. A broader account of this vendor's language model family sits in a separate piece.

Task type, the parameter you must supply

This is the most common source of poor results on first use and the thing setting these models apart from the competition's simplest offerings.

On every call you declare what the vector is for. A document entering the index gets marked differently from a user query, and differently again from text meant for classification, clustering, or judging similarity between two texts.

from google import genai

client = genai.Client()

documents = client.models.embed_content(
    model="gemini-embedding-001",
    contents=chunks,
    config={"task_type": "RETRIEVAL_DOCUMENT", "output_dimensionality": 768},
)

question = client.models.embed_content(
    model="gemini-embedding-001",
    contents=["What is the notice period?"],
    config={"task_type": "RETRIEVAL_QUERY"},
)

The model treats both cases differently, since a query and a document differ in nature. A query is short and interrogative, a document long and declarative, so embedding them identically worsens the match.

The most common mistake is using the same value for both. Search then works, returns results, and is simply noticeably worse, so the problem often goes unnoticed for months.

A separate value serves judging similarity between two texts, when detecting duplicates for instance. Using it for search gives worse results than the pair meant for that purpose, despite seeming intuitively right.

Record which value was used when building the index, and store it alongside the vectors. Changing the task type while adding new documents produces entries incomparable with the rest, and the symptom is simply worse results for part of the collection, with no error and no obvious cause.

One caveat applies to the newer multimodal model, where this parameter does not exist. There you declare the task through a prefix written into the content itself: a query is formatted as task: search result | query: ... and a document as title: ... | text: .... Porting code from the text model therefore means rewriting how you build the text rather than the call. A second difference is easier still to miss: the text model returns a separate vector for each list item, while the multimodal one merges multiple inputs into a single aggregated vector, so embedding many fragments at once is a job for batch mode rather than a list in one call.

The multimodal model

The gemini-embedding-2 model introduced something the competition does not offer at this scope: one space for five kinds of content. The preview opened on 10 March 2026 and general availability followed on 22 April 2026.

Text, images, video, audio, and documents enter a shared space through the same architecture. That means one index serves everything, and comparing a text query against a passage in a recording is an ordinary distance computation, with nothing translated into intermediate text.

The uses where the difference runs largest are specific. An archive of meeting recordings, where a question about a decision finds the moment it was made. A video library, where you search for a scene rather than a file. A document collection whose content lives in visual layout: decks, forms, diagrams.

For that last one, note how many steps it skips. The classic route requires recognising text from the image, reconstructing the structure, and embedding the result, and each of those stages loses some information. Embedding the page as an image skips that whole chain.

The model also accepts four times longer input than the previous generation, eight thousand one hundred and ninety two tokens instead of two thousand and forty eight, which saves work on documents that cannot be sensibly split. That is not an invitation to embed whole documents, though, since a vector averages content, so a document spanning twelve topics stops matching anything.

The per modality limits are hard ones and deserve knowing before you design the index: audio up to a hundred and eighty seconds, video up to a hundred and twenty seconds and at most thirty two frames, a PDF up to six pages and one file per request. Plus a trap straight from the documentation: the audio track of a video file is not processed, so a meeting recording has to be supplied separately as video and as audio.

Truncating dimensions

By default a vector holds three thousand and seventy two dimensions, and the model allows requesting a shorter one, down to a hundred and twenty eight, without recomputation.

The key fact is that the token price does not depend on dimension count. The saving lies entirely on the storage and search side, where it is substantial.

The arithmetic persuades. A three thousand dimension vector takes twelve kilobytes; the same one truncated to seven hundred and sixty eight takes three. Across five million fragments that is the difference between sixty and fifteen gigabytes in the vector database, plus four times faster comparison.

The accuracy loss at around seven hundred dimensions is imperceptible without measurement in most uses. Do perform that measurement, though, since it depends on the collection: fifty real questions with expected answers, the same collection indexed both ways, and a comparison of how often the right document landed in the top five.

A technical caveat: truncated vectors have to be normalised, since cutting disturbs their length. On the text model you do that yourself, for every dimension count other than the full three thousand and seventy two. The multimodal model normalises truncated vectors on its own, so when porting code that step has to be removed rather than copied.

Pricing and two access routes

Billing follows input tokens, and the rate sits mid market: noticeably above the cheapest text providers and below specialist models.

Batch mode halves the price in exchange for results delivered with a delay. For a one off archive indexing that is an obvious choice; for ongoing indexing it is useless.

Choosing between the simpler API and the cloud platform depends on what you need around the model itself. The simpler API gives a key and a call, which suffices for a prototype and a modest application. The platform adds permission management, billing within a cloud project, a choice of data residency region, and processing agreements, which under an enterprise deployment is sometimes a condition of entry.

When budgeting, price indexing and search separately. Indexing a million fragments of five hundred tokens is five hundred million tokens, a one off cost in the tens of dollars. Queries cost far less, since they run to a dozen or so tokens, so indexing dominates the bill rather than user traffic.

From that follows a hint easy to miss: build a vector cache from day one. The same fragment processed twice yields the same result, so storing a hash of the content alongside the vector lets you skip everything unchanged. On an archive updated incrementally that turns costly full recomputation into a cheap top up of the difference, and on a first failed indexing run it saves repeating the entire bill.

Searching recordings in practice

Since video and audio support is the main differentiator here, the whole pipeline deserves showing, because the decisions differ from text.

The first concerns the unit. With text a fragment matches a section or a paragraph; with a recording you must choose a segment length. Too short carries no context, too long dilutes meaning, since five minutes of conversation holds several topics. Segments of tens of seconds with a small overlap work as a starting point.

The second concerns what to keep alongside the vector. A timestamp is mandatory here, since without it a search result points at a file rather than a moment. The user wants to jump to the right minute of a recording, not open an hour of material and hunt through it.

The third concerns cost. Recordings are billed differently from text and the gap is not cosmetic: audio bills by the second and video by the frame, and per million tokens audio lands tens of times above text, with video higher still. An hour of audio runs to tens of cents, so an archive measured in hundreds of hours needs a budget worked out before you start. For meeting recordings, consider a mixed arrangement: a text transcript as the search foundation, with audio embedded only where tone or non verbal content matters.

The fourth, most often skipped, concerns consent. A recording holds specific people's voices, so processing and storing it falls under different rules from a text document. Settle that before building the index rather than after the first question from the legal team.

Hybrid search and the limits of embeddings

Embeddings carry one weakness common to every provider, worth knowing before you build a whole retrieval layer on them.

Meaning based search misses exact matches. A query for an invoice number, an error code, a proper name, or a product identifier may fail to reach the document holding exactly that value, since a string like a product code carries no meaning for the model.

The answer is combining two search methods: the classic one based on word matching and the semantic one. Results from both merge into one list, most simply by position rather than by score, since the two scales are incomparable.

The second limit concerns what embeddings can do at all. They find documents similar in meaning to a question rather than answering it. If an answer requires combining information from three documents or computing something, no embedding model will do that, since it is a task for the layer generating the answer.

The third concerns freshness. Vectors reflect a document's state at indexing time, so changed or withdrawn material must be reprocessed or removed from the index. Without that, search returns content that no longer exists, and that error is harder to notice than an absence of results.

Google embeddings against the alternatives

The decision reduces to one question: what sits in your collection. With notes, documentation, and correspondence, the cheaper text option suffices and you will not notice a difference. With recordings, video, or documents whose content lives in visual layout, the first row does things the others do not do at all.

The second criterion is infrastructure. A team already working in this vendor's cloud gets billing within an existing project, shared permission management, and a region choice, which simplifies compliance more than the price alone suggests.

Common mistakes

The first is omitting the task type or using the same value for documents and queries. Search works and is noticeably worse, so the problem goes unnoticed for a long time.

The second is storing full length vectors on a large collection. Truncating to around seven hundred dimensions gives a four times smaller database with no change in token price.

The third is routing all content through the multimodal variant. Ordinary text gains nothing from it and costs more than a text embedding.

The fourth is embedding whole documents because the model accepts long inputs. A vector averages content, so a document spanning many topics stops matching anything.

The fifth is mixing vectors from different models or different dimension settings in one index. They are not comparable, and the symptom is worse results with no error message.

The sixth is estimating cost from queries alone. In a typical deployment the bill is dominated by one off indexing rather than by traffic.

The seventh is embedding recordings without a timestamp. A search result then points at a file rather than a moment, and the user receives an hour of material along with the task of finding the right minute themselves.

FAQ

Which task type should I set?

For documents entering the index, the value meant for documents; for user queries, the value meant for queries. Separate values exist for classification, clustering, and judging similarity between two texts. Confusing them noticeably worsens accuracy. The multimodal model has no such parameter at all; there the task goes into a prefix written into the content.

What does the multimodal model give me?

It embeds text, images, video, audio, and documents in one shared space, so a single index serves everything and a text query compares directly against a passage in a recording. On a text only archive it gives nothing beyond the text model and costs more.

Does truncating dimensions lower the call price?

No, the token price is the same regardless of dimension count. The saving concerns storage and search speed, and on a large collection it is substantial: truncating to around seven hundred dimensions gives a four times smaller database.

The simpler API or the cloud platform?

The simpler API suffices for a prototype and a modest application. The platform adds permission management, a data residency region choice, and billing within a cloud project, which under an enterprise deployment is sometimes a condition of entry.

Where should computed vectors live?

In any vector database, such as the one covered in the piece on Qdrant. The vendor also offers its own vector search service, convenient when working in the same cloud and binding you to it more tightly than a database you run yourself.

The models are documented on the Google Cloud site, and task types on a separate documentation page.