Understanding PII Detection Guardrails



Why PII Detection Matters

Detect sensitive information before it's exposed
Protect user privacy in data processing
Comply with regulations like GDPR, HIPAA, and CCPA
Prevent data leaks in AI system outputs

Example Response

Example input

Example output

Available Detection Approaches

1. Pattern-Based Detection (Regex)



from guardrails_genie.guardrails.entity_recognition import RegexEntityRecognitionGuardrail

guardrail = RegexEntityRecognitionGuardrail(should_anonymize=True)
result = guardrail.guard("Contact me at john.doe@email.com")

Fast execution
No dependencies
Perfect for well-formatted data (emails, SSNs, phone numbers)
Easy to customize patterns

Prone to false positives
Misses contextual variations
Limited to exact pattern matches

High precision on structured data (emails: 1.0, phone numbers: 0.86)
Lower recall on variations (names: 0.0, addresses: 0.28)

2. Context-Aware Detection (Presidio)



from guardrails_genie.guardrails.entity_recognition import PresidioEntityRecognitionGuardrail

guardrail = PresidioEntityRecognitionGuardrail(should_anonymize=True)
result = guardrail.guard("""
    Patient: John Smith
    DOB: 05/15/1980
    SSN: 123-45-6789
""")

Combines patterns with contextual rules
Supports international formats
Built-in anonymization
Extensible with custom recognizers that are regex, spaCy, or Transformers based

Requires additional dependencies
Slower than pure regex
May need tuning for specific use cases

3. ML-Based Detection (Transformers)



from guardrails_genie.guardrails.entity_recognition import TransformersEntityRecognitionGuardrail

guardrail = TransformersEntityRecognitionGuardrail(
    model_name="iiiorg/piiranha-v1-detect-personal-information",
    should_anonymize=True
)
result = guardrail.guard("""
    Please forward the documents to Sarah Wilson 
    at her new address: 123 Oak Street, Apt 4B
""")

Best contextual understanding
Handles informal text well
Detects subtle PII references
High recall on names and locations

Computationally intensive - especially when training
Requires GPU for optimal performance
Larger deployment footprint

Real-World Usage Examples

1. API Response Filtering

def get_user_profile(user_id: str) -> Dict:
    profile = database.get_profile(user_id)
    
    # Remove PII before returning
    guardrail = RegexEntityRecognitionGuardrail(should_anonymize=True)
    result = guardrail.guard(json.dumps(profile))
    
    return json.loads(result.anonymized_text)

2. Document Processing Pipeline

def process_medical_records(documents: List[str]):
    guardrail = PresidioEntityRecognitionGuardrail(
        selected_entities=["PERSON", "MEDICAL_LICENSE", "PHONE_NUMBER"],
        should_anonymize=True
    )
    
    for doc in documents:
        result = guardrail.guard(doc)
        if result.contains_entities:
            logger.warning(f"Found PII: {result.explanation}")
            doc = result.anonymized_text
        yield doc

3. Chat Bot Safety

def validate_bot_response(response: str) -> str:
    guardrail = TransformersEntityRecognitionGuardrail(should_anonymize=True)
    result = guardrail.guard(response)
    
    if result.contains_entities:
        logger.error(f"Bot leaked PII: {result.detected_entities}")
        return result.anonymized_text
    return response

Evaluation Results

AI4Privacy Dataset for PII Detection Benchmarking

Dataset Characteristics

Contains real-world text samples with diverse PII types
Includes multiple languages and locales
Each sample is professionally annotated with:
Original source text
Masked version of the text
Detailed privacy mask annotations marking PII entities
Language and locale information

Setup

Used 100 randomly sampled validation set entries
Each sample contains multiple PII entities of varying types
Annotations include entity type, value, and position in text
Evaluation metrics track precision, recall, and F1 scores at both entity and overall levels

Model Performance Summary

Detailed Entity-Level F1 Scores

Key observations:

The Transformers model significantly outperforms both Regex and Presidio across all metrics
All models perform well on EMAIL detection (F1 ≥ 0.93)
Regex and Presidio have many entity types with F1 scores of 0.00
The Transformers model achieves perfect F1 scores (1.00) for EMAIL, TAXNUM, and DATEOFBIRTH
The Transformers model shows strong performance (F1 > 0.80) on 9 entity types

Best Practices

Layer Your Approach
Use regex for quick filtering
Apply deeper analysis on flagged content
Combine methods for critical applications
Customize for Your Domain
Add industry-specific patterns
Train on your data formats
Tune confidence thresholds
Monitor and Iterate
Track false positives/negatives
Update patterns based on misses
Regular model retraining

Getting Started

Choose Your Guardrail:
Simple patterns → Regex
Production ready → Presidio
Complex text → Transformers
(Coming soon?) Install Dependencies:

pip install guardrails-genie

Basic Implementation:

from guardrails_genie.guardrails.entity_recognition import RegexEntityRecognitionGuardrail

def check_pii(text: str) -> bool:
    guardrail = RegexEntityRecognitionGuardrail(should_anonymize=True)
    result = guardrail.guard(text)
    return result.safe