» Tag
tokenization
1 posts«August 2026
Unigram: A Rust Codec Turning Bytes Into Single-Token Words for LLMs
Unigram is a Rust crate encoding bytes into words that cost exactly one LLM token each, giving fixed token-budget guarantees over hex or base64.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com