Pinecone, a vector database for semantic search and RAG

Pinecone, a vector database for semantic search and RAG

Pinecone stores embeddings and finds the vectors most similar to a query. You provision no servers, pick no index type, and worry about no replicas, because the service runs serverless and you pay for reads, writes, and storage. That simplifies the start and complicates forecasting the bill.

Why a separate database for vectors

Keyword search finds a document containing a phrase but misses the one saying the same thing in different words. An embedding turns text into a vector of numbers, and the distance between vectors reflects similarity of meaning. A question about returning goods then lands on a paragraph about withdrawing from a contract, despite sharing no words.

You reach for a vector database when there are many such vectors and answers have to arrive in tens of milliseconds. At ten thousand chunks, pgvector on the Postgres you already run is enough. At several million, with a requirement for steady response times, a specialised service gains the advantage, because indexing and scaling reads is its whole job.

It helps to understand that a vector database does not solve quality on its own. Its job is only to find the vectors nearest to a query, quickly. Whether those vectors represent sensible chunks depends on how documents were split and on the embedding model, meaning on decisions made earlier. Swapping databases when results are poor usually changes nothing, because the problem sits one step upstream.

The second reason is division of labour. A team that does not want to maintain another database with backups and upgrades buys the service and pays to be rid of that responsibility.

The billing model and its units

The bill has three components rather than a price per instance.

The most common surprise concerns read units. An unfiltered query over a small dataset consumes one unit, but the same query with metadata filtering over a large index costs several or a dozen. The bill therefore grows not with user count but with how precisely you filter.

Writes behave similarly. Adding one 1536 dimensional vector together with metadata usually costs three to four units, not one. Loading a million chunks in one go therefore carries a real cost worth calculating before you start the import.

Your first index

pip install pinecone

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

pc.create_index(
    name="documentation",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

index = pc.Index("documentation")

Dimension count has to match your embedding model and cannot be changed later. The text-embedding-3-small model from OpenAI produces 1536 dimensions, while smaller open models often produce 384. That difference maps straight onto storage cost, since a 1536 dimensional vector takes four times the space of a 384 dimensional one at the same chunk count.

Writing and searching fit into a few lines.

index.upsert(
    vectors=[
        {
            "id": "terms-4",
            "values": embedding,
            "metadata": {"section": "returns", "language": "en", "version": 3}
        }
    ],
    namespace="production"
)

results = index.query(
    vector=query_embedding,
    top_k=4,
    include_metadata=True,
    filter={"language": "en", "version": {"$gte": 3}},
    namespace="production"
)

You assign vector identifiers yourself, and they should derive from the chunk's origin, for example a document id plus a part number. Reprocessing a document then overwrites the old vectors instead of creating duplicates.

Namespaces and multi tenancy

A namespace splits an index into disjoint spaces. A query always targets one of them, so data belonging to different customers never mixes, even when a filter is wrong.

That beats multi tenancy through a metadata field holding a customer id. A filter can be forgotten by accident, whereas a namespace has to be named deliberately. Namespace operations carry no extra charge.

Be careful with index count, though. Many small indexes perform worse than one larger index split into namespaces, since every index carries its own overhead. The rule is one index per embedding model, with a namespace per customer or per environment.

When removing a customer from the system, deleting an entire namespace is a single operation, which matters when a deletion request arrives.

Namespaces also help when you change how documents are processed. Instead of overwriting production data, you write the new version alongside, compare results on the same question set, and switch the application only once it checks out. The old namespace stays as a fallback, and deleting it a week later is one call. The same mechanism covers embedding model versioning, as long as dimensions match.

Built in embeddings and reranking

Pinecone can compute embeddings itself, so you send text instead of vectors. That removes one pipeline step and one dependency, at the price of tying you to the models the vendor offers.

The more interesting piece is reranking. Vector search returns candidates quickly, but their order can be imprecise, especially when the query contains proper nouns or version numbers. A reranking model takes twenty candidates and reorders them by looking at full text rather than at the vector alone.

The practice runs like this: fetch twenty results, pass them through reranking, hand the top four to the model. Answer quality in a RAG system improves more visibly after this step than after swapping the language model for a larger one, at a fraction of the cost.

Hybrid search

Semantic similarity alone misses precise queries. A question about error ERR_2041 or about version 3.2.1 will find chunks on a similar topic, not necessarily the one carrying that exact number.

The fix combines vectors with keyword search. Pinecone supports sparse vectors alongside dense ones, so a single query merges both signals. The alternative is querying a full text store in parallel and merging results in the application.

A cheaper workaround is detecting the pattern in the query. If the user typed something shaped like an error code or a version number, query your own database with a plain lookup first and treat vector search as the supplement. For many products that handles most cases without adding another layer.

Test this on your own data before considering the topic settled. A set of twenty queries users actually type, paired with the document each should return, exposes the difference between configurations better than any benchmark.

How to measure search quality

The most common mistake in RAG projects is judging the whole system by the model's answer. When an answer is wrong, the model takes the blame, though in most cases retrieval failed and the model simply never received the right chunk.

Separate those two layers with a measurement. Collect thirty real questions and, for each, mark by hand the chunk that ought to appear in the results. Then track one number: in what share of queries the correct chunk landed in the top four.

def hit_rate(dataset, k=4):
    hits = 0
    for question, expected_id in dataset:
        result = index.query(vector=embed(question), top_k=k, namespace="production")
        if expected_id in [d["id"] for d in result["matches"]]:
            hits += 1
    return hits / len(dataset)

That single number says more than intuition does. Below seventy percent there is no point tuning the prompt, because the problem sits in the data or in how documents were split. Above ninety, further retrieval changes yield less and less, and the room for improvement moves into the model instruction.

Gather the question set from real logs rather than inventing it at a desk. Questions written by the system's author echo the documentation being indexed, so they match too easily and inflate the score.

Measure again on every change to chunking. Moving from 400 character chunks to 800 can raise the hit rate by a dozen points or lower it, depending on document type, and without measurement it stays a matter of belief.

Track latency separately. Index response time tends to be stable, but the total grows through embedding the query and reranking. Measure all three segments individually before you start optimising the wrong one.

What it really costs

Take a typical knowledge base: a hundred thousand chunks at 1536 dimensions, ten thousand queries a month.

Storage comes to roughly 0.6 GB of vectors with metadata, so under a dollar. A one off import of a hundred thousand vectors at four write units each is 400 thousand units, under two dollars. Ten thousand queries at five read units each is 50 thousand units, about a dollar.

Usage totals a few dollars, yet on the Standard plan you still pay the 50 USD monthly minimum. The practical conclusion: at small scale you pay mostly for convenience rather than for resources. The break even point against pgvector on the Postgres you already run sits somewhere around a million vectors, or wherever a strict latency requirement appears.

Pinecone against the alternatives

Region matters too. An index in Virginia answers an application in Frankfurt a good hundred milliseconds slower before it computes anything at all. With three calls per user question, that becomes a difference you can feel in the interface.

Migrating between these is not expensive, since vectors move as long as you keep the same embedding model. Changing the model is far costlier, because it requires recomputing the entire dataset.

Before adding a separate database to the stack, though, check whether something you already run can carry it. Elasticsearch handles vectors alongside full text search, giving you a hybrid without another service, and Atlas Vector Search does the same inside the document database your application already queries. A dedicated vector database wins on scale and retrieval quality, not on principle.

Common mistakes

The first is dumping full document text into metadata. Metadata counts towards index size and raises storage cost, while you usually need only an identifier to fetch the text from your own database.

The second is having no update strategy. A document edited a month later leaves stale chunks in the index when identifiers are random. The result: the model receives two contradictory versions of the same clause.

The third is setting top_k to fifty hoping for better results. More candidates means more read units and a longer prompt, while quality improves only up to a point. Fetching twenty and reranking works better.

The fourth is having no test environment. A separate namespace for working data costs next to nothing and lets you test a change in chunking without disturbing production results.

The fifth is ignoring rate limits during import. Loading in huge batches with no pacing ends in errors that leave part of the data unwritten, usually without a clear message in the application log.

FAQ

Does Pinecone have a free plan?

Yes, the Starter plan gives 2 GB of storage, two million write units, and one million read units per month, with no card and no minimum. That covers a prototype and a modest knowledge base. The Standard plan begins at 50 USD per month as a minimum charge.

When should I choose pgvector over Pinecone?

When the data already lives in Postgres, the dataset stays around a million vectors, and you accept tuning the index. You then avoid a separate service and combine vector search with SQL filters in one query. Above that scale, maintenance starts costing more than the subscription.

Can I change the dimension count of an existing index?

No. Dimensions are fixed at index creation and tied to the embedding model. Changing models means a new index and recomputing the whole dataset, so the initial model choice is a long term decision.

How do I integrate Pinecone with a Next.js application?

Make all calls from the server, since the API key must never reach the browser. In practice you build an API route that embeds the query, hits the index, and returns the chunks. LangChain also ships integrations if you are assembling a full RAG pipeline.

Does Pinecone store the original text?

It stores vectors and whatever metadata you send it. Keep the original content in your own database and put only an identifier plus the fields needed for filtering into metadata. That arrangement is cheaper and makes deleting data on request easier.

Billing rules are described in the cost documentation, and pricing sits on pinecone.io.