« All posts

corpus-scrub 0.1.0: Detects and Redacts PII and Secrets in Training Data

corpus-scrub 0.1.0 cleans training data by detecting and redacting PII and secrets.

corpus-scrub 0.1.0 is a tool designed to detect and redact personal identifiable information (PII) and secrets from training datasets. This helps prevent models from inadvertently memorizing sensitive information during training. The application operates locally, ensuring data privacy by not sending any information to external services, thus complying with regulations like GDPR.