PII detection definition
PII detection is the automated identification of personally identifiable information, such as names, email addresses, phone numbers, postal addresses, government ID numbers, and payment card details, inside text, documents, or data, so the system handling it can block, mask, redact, or flag that information before it goes somewhere it should not.
PII detection automatically finds personally identifiable information in text, documents, or data so it can be blocked, masked, or flagged before it spreads. It runs in four common places: before content is published, before text is sent to an AI model or third party, in application logs, and when data is ingested. In Sanity, content is stored as discrete typed fields rather than one blob of rendered markup, so a check can be handed the exact free-text field it concerns instead of a whole page of navigation and boilerplate.
Personally identifiable information is a two-part category rather than a fixed list, which is the single most important thing to understand before building PII detection. NIST Special Publication 800-122 defines it as any information that can be used to distinguish or trace an individual's identity, such as a name, social security number, or date and place of birth, plus any other information that is linked or linkable to an individual, such as medical, educational, financial, or employment information.
That second clause is why PII detection can never be a finished list of patterns. A ZIP code on its own is not PII. A ZIP code next to a birth date and a job title often is, because the combination narrows down to one person. Whether a value counts depends on what sits around it.
Related terms cover overlapping but legally distinct scopes. PHI is protected health information under US healthcare rules, PCI data is payment card information governed by card-industry standards, and personal data is the European term under the GDPR. NIST's glossary notes that personal information and personal data are defined equivalently to PII in several of its documents. A detector usually looks for the same underlying entities regardless of which regime applies to them.
PII detection uses two families of technique, and production systems generally combine them because neither one covers the other's failures.
The first is pattern matching with checksum validation, used for structured identifiers. A credit card number, an IBAN, or a national tax number has a defined format that a regular expression can match, and most carry a check digit that validates the match arithmetically. Payment card numbers use the Luhn algorithm; other identifiers use Verhoeff or ISO 7064 variants. The checksum is what separates sixteen digits from a real card number, and it is the main defense against a format-only false positive. Microsoft Presidio, the most widely referenced open-source implementation, builds each recognizer from a regex, a deny-list, context words, and a validator, then returns a confidence score between 0 and 1 that callers filter with a threshold.
The second is named-entity recognition, or NER, along with newer transformer and large language model detectors, used for free text. A person's name, a street address, or an employer has no fixed format and no check digit. "Robin Park" is a name or a place depending on the sentence around it, so these systems classify spans by context rather than by shape.
The hybrid of the two is the consensus recommendation in recent published work, not one vendor's preference. A July 2025 Scientific Reports paper pairs rule-based natural language processing with a custom NER model for financial documents, and the RECAP evaluation at a NeurIPS 2025 workshop combines deterministic regex rules with context-enriched prompting precisely because standalone rules, standalone NER, and standalone language models each fail in multilingual settings.
PII detection runs at four checkpoints, and they are worth keeping distinct because the consequence of a miss is different at each one.
Before content is published. This catches a real customer email pasted into a draft, a support transcript quoted in a help article, or a sample address left in a screenshot caption. It is the cheapest place to catch something, because nothing has left the building yet.
Before text is sent to a model or a third party. This is the prompt-egress checkpoint. Vendor telemetry on how often sensitive data reaches AI tools varies by an order of magnitude depending on what is measured: Harmonic Security reported on 31 July 2025 that roughly 22% of files and more than 4% of prompts employees sent to generative AI tools contained sensitive information, while Check Point's AI Security Report 2025 put roughly 1 in 80 prompts from enterprise devices at high risk of sensitive data leakage, with a further 7.5% potentially sensitive.
In logs. The OWASP Logging Cheat Sheet is explicit that sensitive personal data should be excluded from logs, and the RAG Security Cheat Sheet extends the same rule to raw queries and model inputs and outputs. Logs are easy to forget and widely replicated.
At ingest. Scanning data as it enters a store means the store never holds what it should not, which avoids a cleanup job later.
PII detection is a classification step, not a guarantee, and published benchmarks show that no current detector gets every entity without also producing false positives. The PII-TRACE benchmark found that no evaluated detector achieved full entity coverage without substantial false positives, particularly over long context or in conversations containing no PII at all. PIIBench reports low span-level accuracy and difficulty with domain and context variability, and Mind the Gap documents precision and recall trade-offs including context-dependent identifiers and numeric confusion under distribution shift.
The failure modes follow from how the techniques work. A detector that flags any person-shaped span will flag the author byline, the public figure quoted in a news story, and the jane.doe@example.com sitting in a code sample. An evaluation of radiology report de-identification published in Radiology: Artificial Intelligence in 2020 shows the same shape of problem in a clinical setting, with medical terms misread as names or addresses, and notes that systems are commonly tuned to favor recall over precision because a missed identifier costs more than a spurious one.
The largest limit is legal rather than technical. Detection is not compliance. Pseudonymized data remains personal data under GDPR Article 4(5), and the European Data Protection Board's pseudonymization guidelines, adopted in January 2025, confirm that data which can still be used to re-identify someone stays in scope. Masking an identifier reduces risk. It does not establish a lawful basis, satisfy retention rules, answer a data subject request, or make an international transfer legitimate.
Structured content improves PII detection by giving the detector a clean, bounded input instead of a page of rendered markup. When content is stored as discrete typed fields and blocks, you know which fields can contain prose at all, so the scan has a defined surface, and you can pass a detector the one free-text field a check concerns rather than navigation, footers, and boilerplate that generate noise and spurious matches. References to taxonomy documents give a classification step a bounded answer space, and document revisions give you the before-and-after of an edit, so a check can look at what just changed.
Sanity is the AI Content Operating System, the intelligent back end for companies building AI content operations at scale, and it offers two mechanically different places to run a check. Schema validation sets a validation property on a field, where the custom method can return a promise and therefore call out to a detection service; warning level surfaces a finding to the editor without blocking, which suits a probabilistic detector, while error level blocks publishing until it is resolved. Validation runs in the Studio, client-side, so it is an editor-facing guardrail. Functions run server-side on document create, update, and delete events regardless of which client wrote the document, and a GROQ filter narrows which changes trigger the check while a GROQ projection narrows which parts of the content get passed in.
A detector that returns a typed verdict rather than prose, the class of model sometimes called a System One model, fits a save-time check well, because the answer space is bounded (does this field contain PII, and with what probability) and the check runs constantly.
Discover More with Sanity
Now that you've learned about PII detection, why not start exploring what Sanity has to offer? Dive into our platform and see how it can support your content needs.
Last updated: