pdfmuse: A Deterministic PDF Parser That Failed on Real Resumes
pdfmuse's deterministic PDF parser looked solid until real resumes exposed a silent 10% failure rate from unhandled form XObjects.
The most fragile step in RAG pipelines is often the PDF-to-text conversion: the same file parsed twice can produce different output, which quietly breaks embeddings and evaluation numbers. pdfmuse is built as a deterministic PDF/DOCX parser with a single Rust core and byte-identical Python/Node/WASM bindings, deliberately avoiding probabilistic models in its main path. Benchmarks claim ~7.7x speedup over PyMuPDF and ~150x over pdfplumber with full character coverage.
The more instructive part is what happened after a clean test suite — snapshot tests, fuzzing, cross-binding parity — passed on synthetic fixtures. Running the parser on 80 real resumes revealed that 10% returned completely empty output with no warning at all. The cause: tools like Canva render text inside form XObjects invoked via a Do operator, which the content-stream interpreter never handled, and a lazy-loading optimization skipped those object bodies entirely for speed.
Fixing the interpreter to recurse into form XObjects dropped the zero-character failure rate from 8/80 to 0/80. A separate finding showed that deserializing the full intermediate representation back into Python ate most of the claimed speed advantage; returning plain strings directly from Rust for text-only calls restored the performance gap. The takeaway generalizes well beyond this project: a green test suite on clean sample files tells you nothing about your real data distribution — always shadow-test any parser swap against your own corpus before trusting it.