Sentry's New AI Model Cuts Issue Grouping Errors in Half

Sentry groups incoming errors into issues first via lexical fingerprinting, then via an ML model that merges semantically similar stacktraces when fingerprinting fails. The company has now rolled out v2 of this model, cutting duplicate issue creation by 20% while halving the overgrouping (incorrect merge) rate — from 8% down to 4% overall, with per-platform rates that previously spiked to 30-60% now capped between 2-15%.

The post illustrates the difficulty with two nearly identical stacktraces sharing the same timeout error but different root causes — one from a database query, the other from a GPU-based ML call — which v1 incorrectly merged but v2 correctly separates. v2 is built on lightonai/modernbert-embed-large, trained on hundreds of thousands of stacktrace pairs labeled using Claude Sonnet 4.5, encoding years of tribal knowledge from Sentry's internal grouping experts into a scalable labeling pipeline.

Beyond accuracy, the team optimized inference: using Matryoshka Representation Learning, embeddings were truncated from 768 to 64 dimensions, sacrificing only 2% accuracy while delivering 4x faster HNSW lookups, 20x faster insertions, and 12x less database storage. Overall, v2 runs 6x faster than v1 with lower GPU memory use, and Sentry has open-sourced its training and evaluation infrastructure — offering a concrete engineering case study in improving grouping quality for error-tracking systems.