Cohere, the vendor that bet on search

Cohere, the vendor that bet on search

Most companies in this field race on conversational models. Cohere took another path and built its position on three less spectacular things: embeddings, reranking, and deployments inside a customer's own infrastructure.

That means its products are rarely what you want for building a conversational assistant and often what you want when an assistant returns off target fragments and nobody knows why.

Reranking

This is the company's most distinctive product and deserves its own treatment, since it solves a specific problem.

Vector search returns fragments semantically similar to a question, but vector similarity is an approximation. A reranking model scores the question and fragment pair together, so it sees more than the distance between vectors.

import cohere

client = cohere.ClientV2(api_key=os.environ["COHERE_API_KEY"])

results = client.rerank(
    model="rerank-v3.5",
    query="how do I download an invoice copy",
    documents=fragments_from_store,
    top_n=5,
)

The arrangement runs like this: the vector database returns twenty candidates, the reranker scores each, and the best five reach the prompt. It costs one extra call and usually improves accuracy noticeably.

Measure whether it actually improves things in your case, though. Across thirty real questions, compare accuracy before and after. If the difference is slight, you save a call and the latency, and that applies to every query.

A cheaper alternative raises the number of returned fragments while tightening filters. Often the problem lies not in the ordering but in the right fragment being absent from the returned set, and then reranking has nothing to fix.

Embeddings and languages

This vendor's embedding models perform well outside English, and that is their practical advantage.

A model trained mainly on English data returns worse matched fragments in other languages, and diagnosing that misleads, since it looks like the vector database's fault. A multilingual model removes that cause and is usually the cheapest improvement to search quality.

embeddings = client.embed(
    model="embed-v4.0",
    texts=fragments,
    input_type="search_document",
)

The input type parameter matters here and often gets skipped. Embeddings for documents and for queries are computed differently, since they play different roles. Using the same type for both lowers accuracy and nothing reports it.

Remember too that changing the embedding model requires recomputing the whole collection. Vectors from different models are not comparable, so test two models on a sample rather than after indexing a million fragments.

Newer generation models also handle images in the same space as text, letting you search documents containing charts and screenshots without separate processing.

Generative models

Beyond search the vendor has generative models built for working with documents and for calling tools.

Their distinguishing feature is built in citations. A model answering from supplied fragments also returns which fragment each sentence came from. That solves a problem other models require you to work around with instructions and parsing.

In enterprise applications that property is sometimes decisive, since an answer without a source is useless in many contexts. A user must be able to check where information came from, and with internal documents that is often a formal requirement.

In fairness, the vendor's models do not lead general quality benchmarks. Their point lies in combining decent quality with citations, a price below the leaders, and the option of deployment inside your own infrastructure.

Deployment in your own infrastructure

That is the second pillar of this company's position and the reason it appears in conversations about deployments in banking and government.

The models can run in a customer's private cloud or at the major cloud providers with control over keys retained. Data then never leaves the chosen region, which under compliance requirements is a condition rather than a preference.

Two things need separating here, because they bill differently. Dedicated instances operated by the vendor carry a published hourly and monthly rate: from 4 dollars an hour for the smaller performance tier of the embedding model to 10 dollars an hour for the larger tier of the reranker, meaning 2,500 to 6,500 dollars a month per instance. Running the models inside your own infrastructure is a separate agreement and a custom quote.

An agent platform for internal deployments is also available, combining search over company documents with tool calling. Prices for that part are not published, so budget planning requires going through a sales conversation.

That opacity is a real drawback. Comparing cost against alternatives then takes time and several meetings, while with vendors publishing prices an estimate takes a quarter of an hour.

The full search pipeline

It helps to see how these three products form one path, since only then does it become clear where each fits.

vector = client.embed(
    model="embed-v4.0",
    texts=[question],
    input_type="search_query",
).embeddings.float[0]

candidates = store.search(vector, limit=25, filter={"language": "en"})

best = client.rerank(
    model="rerank-v3.5",
    query=question,
    documents=[c.content for c in candidates],
    top_n=5,
)

answer = client.chat(
    model="command-a-03-2025",
    messages=[{"role": "user", "content": question}],
    documents=[candidates[r.index].content for r in best.results],
)

Four steps, each replaceable separately. Embeddings turn the question into a vector, a vector database such as Qdrant returns candidates, the reranker picks the best, and the generative model composes an answer with citations.

The candidate count before reranking is a parameter worth thinking through. Twenty five gives the reranker something to choose from and raises no cost, since billing runs per search. Raising it to a hundred rarely improves the result and lengthens response time.

The metadata filter applies before search, so it narrows the set rather than sieving results afterwards. In a multi customer application that is a matter of correctness rather than accuracy, since a missing owner filter means a data leak.

Pricing

Billing reranking per search rather than per token is unusual here and worth remembering when estimating. One search, however, means one query with at most a hundred documents, and a document longer than five hundred tokens is split by the vendor into chunks counted as separate documents. Up to a hundred candidates the bill therefore follows query count, and above that threshold it climbs in steps, because a single call counts as several searches.

When building document search, the largest one off cost is usually indexing, meaning computing embeddings for the whole base. A million fragments at that rate is a cost of tens of dollars, and changing models brings it back.

Rates get adjusted, so check the current price list at source before budgeting for a year rather than relying on articles.

Citations in practice

Built in citations are a feature easy to overlook in a description and one that changes what the finished product looks like.

The model returns an answer alongside which passage rests on which document. In an interface that becomes links beside sentences, letting a user check a source without asking anybody.

The value is double. First, a user trusts an answer more when they can see where it came from. Second, and more importantly, citations reveal the case where the model answered correctly from its own knowledge rather than from the supplied documents. An answer attributed to no fragment is a warning sign.

Remember, though, that a citation confirms a sentence's origin rather than its truth. If the source document is out of date, the answer will be correctly cited and wrong. That argues for keeping the index fresh rather than for relaxing.

Where answers reach customers, show citations always, even when the interface grows denser for it. An answer without a source and an answer with one are two different products in terms of trust.

When not to reach for it

It deserves saying plainly when this vendor is the wrong choice, since the list is specific.

For building a conversational assistant without a search element, other vendors hold stronger models. If the task is reasoning, writing code, or holding a long conversation, the advantage sits elsewhere.

On a prototype where time to a first working result matters, adding a separate vendor for embeddings and reranking complicates configuration. Then taking everything from one vendor and returning to optimisation once there is something to optimise is simpler.

With a set of a few thousand fragments, reranking rarely makes a difference, since vector search hits within a small set anyway. It earns its keep at tens of thousands, where similar candidates are many.

Cohere against the alternatives

The practical conclusion is that these vendors need not be chosen exclusively. A common arrangement takes embeddings and reranking from one vendor and answer generation from another, since those are three independent steps and each can be staffed separately.

That split carries a practical advantage too. Changing the generative model requires no embedding recomputation, and changing the embedding model does not touch the generation layer. With one vendor for everything, migration grows larger than it should.

Do work out, though, whether splitting across three vendors costs more than it gives. Three contracts, three keys, three places to check during an outage, and three price lists to track are a real operational overhead. On a small project one vendor for everything is often cheaper to maintain, even when each element separately comes out worse.

The line runs where search quality starts deciding the product's usefulness. In an assistant answering questions about documentation, wrong fragments are a problem visible to the user on every question, so optimising that stage repays faster than the cost of maintaining a second vendor.

Common mistakes

The first is skipping the input type when computing embeddings. Documents and queries need different types, and using the same for both lowers accuracy with no message about it.

The second is adding reranking without measuring. It is an extra call on every query, so check whether the accuracy gain justifies the latency and the cost.

The third is changing the embedding model without recomputing the collection. Old and new vectors sit in different spaces, so results turn random and nothing reports it.

The fourth is comparing models on somebody else's benchmarks. Outside English the differences sometimes invert those measured on English data, so a test on your own fragments is mandatory.

The fifth is budgeting without accounting for indexing. The one off cost of first indexing is sometimes larger than a month of running the system.

The sixth is assuming better answers follow from changing the generative model. If the right fragment is absent from the context, no model answers correctly, and diagnosis must start at retrieval.

FAQ

What is Cohere best suited to?

Building search over your own documents: embeddings that handle languages beyond English, a reranking model, and generative models with built in citations. For building a conversational assistant without a search element, other vendors perform better.

What is reranking?

The second stage of search. A vector database returns candidates by vector similarity, and a reranking model scores the question and fragment pair together, judging more accurately what actually answers the question. It costs one extra call and usually improves results.

What does it cost?

Embeddings run twelve cents per million text tokens, reranking two dollars per thousand searches on the cheaper model variant, and generative models a few dollars per million tokens. Private deployment prices are not published and require a sales conversation.

Do the models handle languages other than English?

Better than models trained mainly on English data, particularly on embeddings. That is the commonest reason European projects reach for this vendor, though a comparison on your own fragments is still worth running.

Can it be deployed in my own infrastructure?

Yes, and that is one pillar of this offering. Models run in a private cloud or at the major providers with key control retained, which under compliance requirements is sometimes a necessary condition.

Documentation sits on the project site, and a rate comparison appears in a pricing roundup.