Milvus, a vector database built for billion scale

Milvus, a vector database built for billion scale

Milvus is an open source vector database under Apache 2.0, developed within the LF AI & Data Foundation and designed for datasets where vector counts run into the billions. What sets it apart is separating storage from compute, letting each layer scale on its own. Version 3.0 landed on 29 July 2026, bringing queries against data that lives outside the database and schema changes without downtime.

When you actually need a vector database

I will start with a question that saves considerable effort when asked before a deployment rather than after. You need a vector database when the vector count stops fitting in process memory, or when you need operations an index library does not provide: metadata filtering, record updates, durability across restarts, access control.

Below roughly a hundred thousand vectors none of those requirements usually applies. A plain in memory array and a matrix multiply through a numerical library answer faster than any database, because there is neither serialisation nor network traffic. At a million vectors the memory arithmetic begins: a thousand dimensions in single precision floats is four kilobytes per vector, so four gigabytes for the raw set alone, with no index and no metadata. Past that threshold the decision makes itself.

Milvus aims at the upper end of that range. If your set holds half a million vectors and you expect no growth, this database will work correctly, but you will pay for it in operational complexity you do not need. Simpler options, pgvector inside a database you already run, or Chroma in a prototype, are the sensible pick then.

Three deployment modes and the real difference between them

Milvus Lite is where it makes sense to start, because the entire deployment layer disappears. You install the package and pass a file path:

pip install -U "pymilvus[milvus-lite]"

from pymilvus import MilvusClient

client = MilvusClient("./milvus_demo.db")

client.create_collection(
    collection_name="documents",
    dimension=768,
)

The interface matches the server version, so moving from a prototype to a cluster comes down to swapping the argument for a server address. That is this project's strongest card and the reason to begin here rather than standing up containers on day one.

Lite in its third generation, released in May 2026, was rewritten from scratch in pure Python and looks nothing like most older material describes. Instead of wrapping the C++ core it runs a log structured storage engine with a write ahead log, immutable Parquet segments, and indexes from the FAISS library. Available indexes are FLAT, BRUTE_FORCE, IVF_FLAT, IVF_SQ8, HNSW, HNSW_SQ, and AUTOINDEX, plus sparse SPARSE_INVERTED_INDEX and scalar INVERTED. Partitions, aliases, named databases, iterators, group by search, and hybrid search are all there.

What Lite still lacks deserves checking before you commit. There is no authentication, no users, no roles, and no transport encryption, so the local server does not go on a network. There are no binary, half precision, or integer vector fields, and no product quantisation indexes. One data directory serves one process, and writes to the same collection have to be serialised. Python 3.10 or newer is required, and the system can be Linux, macOS, or Windows as long as dependencies such as faiss-cpu and pyarrow have wheels there. Database files from the previous engine do not open in the new one and there is no automatic migration, only an export to import files.

Standalone is a single container with the full index set. It handles tens of millions of vectors and underpins most deployments I have seen. Distributed splits query, data, index, and coordination nodes, so read capacity and index capacity scale independently, but it requires Kubernetes and somebody to keep it running.

Index choice, where the real decisions happen

Index choice affects cost and latency more than the choice of database. Three families are worth understanding.

FLAT scans every vector. It gives perfect accuracy and is the slowest. It earns its place at sets up to a few tens of thousands of vectors, or as the reference point you measure faster indexes against.

The IVF family partitions the space and searches only some regions. The nlist parameter says how many regions to split into, and nprobe how many to visit per query. Raising nprobe improves recall and lengthens response time, making it the simplest dial for trading quality against speed after deployment, with no index rebuild.

HNSW builds a multi layer neighbourhood graph and is the default choice for most workloads whose data fits in memory. You pay for it in memory use noticeably above the raw vector size. DISKANN instead keeps the graph on disk and handles sets that do not fit in memory, at a latency that depends on the storage medium. On NVMe drives the difference is often acceptable; on slower ones it shows immediately.

Quantisation deserves separate mention, meaning storing vectors at lower precision. Version 2.6 introduced one bit quantisation, for which the project team reported a 72 percent reduction in memory use with recall preserved. That figure comes from vendor material, so treat it as a starting point for your own measurement rather than a guarantee. The mechanism is real, though, and on large sets it is quantisation, not a change of database, that delivers the biggest saving.

What is new in Milvus 3.0

The 29 July 2026 release is one of the larger ones in the project's history, and several changes are worth knowing because they change how you design the system.

External collections let you query data sitting in Parquet, Lance, and Iceberg formats without copying it into the database, with incremental refresh. That inverts the previous assumption that everything had to be imported first. For datasets that already live in a lakehouse, the whole copy step and the burden of keeping two copies consistent disappear.

Flexible schema allows adding, backfilling, and dropping columns with no downtime and no collection rebuild. Anyone who has tried to add a field to a collection holding several hundred million vectors under production traffic will value this more than everything else combined.

Sparse indexes were overhauled, adding SINDI, Block-Max WAND, and Block-Max MaxScore with better compression. Sparse representations carry keyword matching, so this is the layer hybrid search rests on.

Aggregations and faceted search arrived too, returning top facet values alongside COUNT and AVG in a single request. That previously meant a separate query against a different database. TEXT fields now store text beyond 64 kilobytes next to vectors, with values above that threshold moved into separate files while the column keeps only a reference. That feature depends on Storage V3, covered below. FAISS passthrough in turn accepts an arbitrary index factory string from that library, reproducing a research recipe directly.

Two changes need attention during an upgrade. Storage V3 is disabled by default and is enabled manually through common.storage.useLoonFFI. Rolling back from 3.0 to 2.6 remains possible as long as you have not enabled Storage V3 or other format changing features, so leave those off for the first few weeks if you want a way back. GPU images moved to CUDA 12.9, which ends Ubuntu 20.04 compatibility in the GPU variant.

Hybrid search and filtering

Vector similarity alone rarely suffices in a product. A user searching for "lease agreement 2025" expects the document carrying exactly that year to rank above a semantically similar document from another year, and dense vector matching does not guarantee that.

The answer is combining results from dense and sparse vectors with a reranking pass at the end. Milvus 3.0 adds a composable reranking chain where successive stages rescore and reduce the candidate set.

from pymilvus import MilvusClient, AnnSearchRequest, RRFRanker

client = MilvusClient(uri="http://localhost:19530")

dense_request = AnnSearchRequest(
    data=[query_vector],
    anns_field="dense_vector",
    param={"nprobe": 32},
    limit=50,
)

results = client.hybrid_search(
    collection_name="documents",
    reqs=[dense_request, sparse_request],
    ranker=RRFRanker(),
    limit=10,
    output_fields=["title", "year"],
)

Metadata filtering deserves its own note, because that is where performance most often degrades. You pass a filter expression through the filter parameter, and the database decides whether to apply it before or after searching the index. With a very narrow filter, one removing 99 percent of the set, searching an approximate index stops paying off and a plain scan of what remains is often better. With a broad filter the reverse holds. If response times look unpredictable, check filter selectivity before you start tuning index parameters.

Milvus against the alternatives

The choice between them is rarely settled by a feature list, since for a typical workload all of them do the same job. It is settled by scale and by who maintains the deployment. Milvus in distributed mode needs Kubernetes and genuine operational knowledge. Qdrant at ten million vectors goes up in an hour and simply works; at a billion you start fighting it. Pinecone removes the maintenance layer entirely and charges for it on a bill that grows faster at large scale than intuition suggests.

Practical advice: when picking a database, size it against the vector count you expect in two years rather than today's. Migrating between these systems is feasible, since they all accept the same vectors, but rewriting the query, filter, and reranking layer takes longer than people assume.

Common mistakes

The most frequent is failing to load a collection before searching. Milvus separates durable state from in memory state, so a collection persisted to disk is not automatically ready to serve queries. The symptom is a collection not loaded error, often misread as a connection problem.

The second is expecting writes to be visible immediately. Data lands in the write ahead log first and reaches the index afterwards, so a vector inserted a moment ago may be absent from the next query's results. An integration test that inserts and immediately searches needs a consistency level or a short wait.

The third is an unconsidered similarity metric. Inner product and cosine distance produce the same ranking only for normalised vectors. An embedding model that does not normalise its output, paired with the wrong metric, yields results that look plausible and are systematically skewed. Check it against ten known cases before you accept the quality.

The fourth is treating measurements from Lite as a forecast for a cluster. Lite is a single Python process with writes to one collection serialised, and it computes keyword matching statistics per segment rather than globally. Even on the same HNSW index, prototype numbers describe nothing about how a distributed deployment behaves.

The fifth is underestimating memory. On top of the raw vector size comes index overhead, which for HNSW can rival the size of the data. A cluster sized from vector count alone runs out of memory on the first full load.

Fitting it into an existing stack

In a typical retrieval augmented generation system Milvus is the storage layer with an orchestration library sitting above it. Both LangChain and LlamaIndex ship integrations, so swapping the database underneath reduces to changing a vector store class.

One design decision recurs in every such deployment and is worth making deliberately up front: where to keep document content. Putting the full text in a field next to the vector is tempting, since it simplifies the code and one query returns everything. The cost is collection size and the fact that every content edit requires updating a record in the vector database. The alternative keeps only identifiers there and fetches content from PostgreSQL or object storage. For longer, frequently edited documents the second approach ages better, and the TEXT fields above 64 kilobytes introduced in 3.0 mean the first is no longer technically ruled out.

FAQ

Is Milvus free?

Yes, Milvus ships under Apache 2.0 and can be used commercially at no cost, including in distributed mode. What is paid is the managed Zilliz Cloud service, run by the company behind the project, along with commercial support.

How does Milvus Lite differ from the full version?

Lite runs as a library inside a Python process and writes to a local file. It shares the interface and, since its third generation, also FLAT, IVF, and HNSW indexes, sparse and scalar indexes, partitions, and aliases. It has no authentication, users, or roles, serves one process per data directory, and is meant as an environment for prototypes and notebooks rather than production.

Does Milvus run on Windows?

Milvus Lite in its third generation is pure Python and runs on Linux, macOS, and Windows wherever its dependencies have wheels; the project runs continuous integration for Windows on Python 3.10. You run the full server version on Windows through Docker or the Linux subsystem, which in practice means running it in a container.

Which index should I start with?

HNSW, provided the data fits in memory. It gives a good recall to latency ratio without tuning. Once the set stops fitting in memory, move to DISKANN or add quantisation rather than immediately enlarging the cluster.

Is upgrading from Milvus 2.6 to 3.0 safe?

Rolling back to 2.6 stays possible as long as you do not enable Storage V3 or other format changing features. Storage V3 is off by default and new index versions require deliberate activation, so the safe path is to upgrade first without those options.

Release details are covered in the Milvus release notes, and the source lives in the project repository.