nibble: Diagnose and Fix Redis Memory Waste, No Install Required

nibble is a diagnostic tool that connects to a running Redis instance, samples its keys, and reports exactly where memory is being wasted - missing TTLs, verbose JSON encodings, un-acked queue backlogs - then proposes and, if asked, applies the fixes. It ships as a single zero-install script with its own minimal bundled Redis client.

Across measured components (telemetry, agent run state, embeddings, conversation transcripts, dedup keys) the project reports an 8.6x overall memory reduction, from 919 GiB down to 107 GiB, which translates to a drop from roughly $11.9k to $1.4k per month at ElastiCache list pricing. The gains come from tuning shard width against jemalloc's allocation size classes, replacing repeated JSON field names with schema-based packing, rounding time-series values to integers for better compression, and correcting TTL and eviction settings.

The project also documents specific Redis internals: hash field templates and sharding are mutually exclusive optimizations that shouldn't be combined, the real embstr allocation limit is a combined key+value budget rather than the commonly cited 44-byte rule, WAITAOF under everysec mode can silently stall throughput, t-digest sketches lose their advantage over RedisTimeSeries below roughly 4,700 samples per series, and bimodal latency distributions can push p99 estimation error as high as 11x. It also publishes techniques that failed and a retracted benchmark number, showing that binary vector quantization recall depends heavily on a corpus's intrinsic rank rather than the quantizer itself.