Presidio, detecting and anonymising PII

Presidio is a library and a set of services for finding personal data in text, images, and tables, and for masking, encrypting, or replacing it. It runs locally, under the MIT licence, and sends nothing outside.

The typical use looks like this: you hold customer tickets, call transcripts, or application logs, you want to analyse them or send them to a language model, but you cannot hand over names, card numbers, and addresses. Presidio sits in that spot as a filter.

First thing to know: the project changed owners

The name "Microsoft Presidio" is now misleading, and that is the most important fact before you start.

The project is moving from Microsoft to an independent, community-governed organisation called Data Privacy Stack. The repository now lives at data-privacy-stack/presidio, and the technical steering committee includes people from outside Microsoft.

What stays unchanged: the MIT licence, existing functionality, the programming interfaces, and integrations with Azure services. Code that works today will keep working.

What needs your attention: container images. New releases publish to the GitHub registry under ghcr.io/data-privacy-stack/presidio-*. The old mcr.microsoft.com addresses stay available but stopped pointing at the newest release, so a latest tag in that location freezes you on an old version with no warning at all.

This is exactly the kind of quiet change you notice six months later, when recognisers added in the meantime turn out to be missing. If your deployment configuration holds an mcr.microsoft.com address, change it now.

What Presidio is made of

The project splits into several pieces usable separately, and that separation has a practical point.

The analyser finds fragments in text that look like personal data and assigns each a type and a confidence score. It returns positions in the text, not modified text.

The anonymiser takes the analysis result and performs an operation on it: replacing, masking, hashing, encrypting, or removing. Splitting those two steps lets you inspect what was found before anything changes.

Separate components handle images, where text first goes through character recognition and detected fragments get painted over, and structured data, where the operation runs on whole columns rather than individual occurrences.

Each of those pieces runs either as a library inside a Python process or as a service in a container. The library version is faster, since there is no network hop; the service version is more convenient when the rest of your application is written in another language.

Installation and first run

Code

Bash

pip install presidio-analyzer presidio-anonymizer
python -m spacy download en_core_web_lg

You download the language model separately, and that is the step most often forgotten. Without it, detection of names and organisations will not work, since it rests on a named entity recognition model rather than on regular expressions.

Code

Python

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

text = "Customer contact: anna.smith@company.com, phone 555 234 567."

results = analyzer.analyze(text=text, language="en")
for r in results:
    print(r.entity_type, r.start, r.end, r.score)

print(anonymizer.anonymize(text=text, analyzer_results=results).text)

The output looks roughly like this:

Code

TEXT

EMAIL_ADDRESS 18 40 1.0
PHONE_NUMBER 48 59 0.75
Customer contact: <EMAIL_ADDRESS>, phone <PHONE_NUMBER>.

Note the confidence scores. The email address gets a one, since it passed format validation. The phone number gets 0.75, since a digit string in that shape is sometimes an invoice number or an order identifier. That figure is your main dial: the cutoff threshold decides the balance between data slipping through and text mangled beyond use.

Languages and the default configuration

The default configuration is English, and that is the source of most disappointment on a first run against non-English text.

Recognisers fall into two groups. Universal ones, based on patterns and checksums, work regardless of language: email address, card number, IP address, crypto wallet address. Language and country specific ones require the right model and the right configuration: names, addresses, national identifiers.

The country set grows with every release, though it spreads unevenly. Ready recognisers exist today for the United States, the United Kingdom, Spain, Italy, Poland, Singapore, Australia, India, Finland, Korea, Nigeria, the Philippines, Canada, Sweden, South Africa, Thailand, Turkey, and Germany, except that the number of types per country runs from one to well over a dozen.

Poland, for example, has exactly one: PL_PESEL, checked by pattern, surrounding words, and a checksum. That recogniser is bound to the Polish language, so it speaks up only once you run analysis with language="pl" and a Polish model loaded, not in the default English configuration. Plenty of other national identifiers have no ready recogniser at all, so you write them yourself, covered below.

A country filter on predefined recognisers arrived as well. That is a practical addition, since loading dozens of recognisers from around the world into a system serving one market raises false positives and slows analysis for no gain.

A custom recogniser

This is where Presidio earns its keep on a company project, since the default types never cover your data.

Code

Python

from presidio_analyzer import Pattern, PatternRecognizer

national_id = PatternRecognizer(
    supported_entity="NATIONAL_ID",
    patterns=[Pattern(name="national_id", regex=r"\b\d{11}\b", score=0.4)],
    context=["national id", "identity number"],
)

analyzer.registry.add_recognizer(national_id)

Two things in that code matter more than the regular expression itself.

The starting score of 0.4 is deliberately low, since eleven digits is also a phone number with a prefix or an identifier in a warehouse system. The context list raises that score when a nearby word matches. This is context enhancement, and it separates Presidio from plain regular expression searching.

For numbers carrying a checksum, and most national identifiers do, it pays to go a step further and write a class with a validation method. A correct checksum lifts the score to one, a wrong one drops it to zero, and false positives fall to practically nil. The same mechanism covers card numbers, where the Luhn checksum gets verified, and the built in national identifier recognisers are put together the same way, so their source doubles as a template to copy.

Anonymisation operations and picking the right one

Operation	What it does	Reversible	Use it when
Replace	Inserts a type label	No	Text goes to a language model
Mask	Leaves the last few characters	No	A support agent interface
Hash	A cryptographic digest	No	Counting unique people
Encrypt	Symmetric cipher with a key	Yes	The original must be recoverable
Remove	Cuts the fragment out	No	Publishing a data set

Choosing between hashing and encryption comes down to one question: will anyone ever need to recover the original value. If so, a hash is a dead end and encryption is required, with care around the key, since leaking it voids the whole protection.

Note also that a hash is not anonymisation under privacy regulation when the space of possible values is small. A hashed phone number can be recovered by computing digests of every nine digit number, which takes minutes on an ordinary computer. Salting helps, provided the salt does not sit next to the data.

Images and tabular data

Text is the most common case, but the other two follow their own rules, worth knowing before a leak turns out to have taken a route nobody considered.

For images the pipeline runs like this: character recognition extracts text along with positions, the analyser scores that text, and detected rectangles get painted over. Effectiveness therefore rests mainly on character recognition quality rather than on Presidio itself. A document photographed at an angle, a low resolution scan, or handwriting yields text with typos that no recogniser will match.

The practical consequence is that with images the confidence threshold deserves lowering further than with text, and the result still deserves review. Better to paint over two rectangles too many than to leave an identity card number visible.

Structured data is a separate story, since the decision there happens at column level rather than per cell. A column named email almost certainly holds addresses, and analysing every row separately makes no sense. Presidio can infer a column type from a sample of values and then apply one operation to the whole thing, which across millions of rows is the difference between minutes and hours.

The caveat concerns free text columns, customer support notes for instance. Header based inference fails there and you fall back to cell by cell analysis, accepting that it will be the slowest part of the whole job. Parallelise it then and measure on a sample before running it across the full set.

Presidio against the alternatives

Option	How it runs	Cost	Pick it when
Presidio	Locally, library or container	Free, MIT licence	Data cannot leave your infrastructure
Vendor cloud services	A call to an external interface	Per thousand characters	Low volume, no team to maintain it
A language model for detection	A prompt to a model	Per token	Unusual data with no clear pattern
Your own regular expressions	Code in the project	Developer time	One well defined format

The third row deserves honest consideration, since it is tempting. A language model handles data no pattern can describe, a health description allowing identification of a person for instance. The price is a cost per document, latency, and no repeatability: the same text can produce different results. For millions of records it does not scale.

The sensible arrangement in practice is Presidio as the first layer for everything with a pattern, and a language model only for the rest. A model running on one percent of documents costs one percent of a model running on all of them.

Presidio in front of a language model

The most common use today is a filter between company data and an external model.

The arrangement is simple: before sending a prompt you push it through the analyser and the anonymiser, send the version stripped of personal data, and substitute the original values back into the response. That last step needs encryption or your own replacement map, since plain label substitution is irreversible.

This layer solves a different problem from mechanisms such as Guardrails AI or NeMo Guardrails. Those watch what the model answers. Presidio watches what the model gets to see at all. A serious deployment needs both, since a model can after all reveal data it received earlier in another context.

Building such a chain in LangChain has a ready integration, though your own wrapper around two function calls is no harder and gives full control over exactly what leaves the building.

One pitfall here is not obvious. Anonymisation changes text length, so if the same flow computes character positions, for highlighting fragments in an interface for instance, they stop matching afterwards. Store the offsets alongside the analysis results.

Common mistakes

The first is relying on the default confidence threshold. The default is a compromise tuned for English text and on your data will almost certainly need shifting one way or the other.

The second is a missing language model. Without a downloaded named entity recognition model, names and organisations pass undetected, and the library does not report that as an error.

The third is treating the result as certain. Presidio is a statistical tool and lets some data through, particularly when written unusually, with typos, or in a foreign format. A data set headed for publication needs human review.

The fourth is hashing where reversible pseudonymisation was needed. Discovering that after the originals were deleted means losing the data.

The fifth is leaving the old container registry address in place. The latest tag on mcr.microsoft.com no longer points at the newest release, so a deployment quietly sits on an old version.

The sixth is anonymising text while skipping metadata. A name removed from a document body but left in the file name, in the file properties, or in a path inside a log has not been removed.

FAQ

Is Presidio free?

Yes, under the MIT licence, with no fees and no volume limits. The cost is compute on your side plus the time to tune recognisers, since the default configuration rarely fits a specific data set.

Does Presidio handle languages other than English?

Partly. Pattern based recognisers such as email address or card number work regardless of language. Detecting names requires the matching spaCy model. Eighteen countries have predefined recognisers, though each loads only when analysis is set to that country's language, and identifiers outside that set have to be added as custom recognisers.

Does Presidio send data anywhere?

No. All analysis happens locally, in your process or your container, and that is the main reason it gets chosen over a cloud service. The exception is when you deliberately configure an external model as the detection engine.

Is Presidio enough for privacy regulation compliance?

Not by itself. It is a technical tool, and compliance depends on the legal basis for processing, retention, access control, and everything else. It does help with pseudonymisation, which regulation names as a risk reducing measure.

How does anonymisation differ from pseudonymisation?

Anonymisation is irreversible, and afterwards the data stops being personal data. Pseudonymisation swaps values for substitutes but allows recovery with a key, so the data remains protected material. Presidio does both, depending on the operation you pick.

Documentation sits on the project site, and the code and releases in the GitHub repository.