Reading AI Memory OSS as a Backend System, Not a Prompt

AI memory systems are usually judged by their prompts, embeddings, and vector search — but production readiness depends on less visible contracts: when raw input becomes durable, where slow LLM calls are isolated from request/DB scope, what derived 'memories' are actually grounded in, and how the system recovers when the vector index and database drift apart. This piece uses the pinned source and change history of Honcho, an open-source AI memory project, to build a question map for evaluating such systems as backend infrastructure rather than feature demos.

The ten questions cover API request boundaries, the separation between raw messages and LLM-derived claims, logical work identity in queues, what LLMs are allowed to decide versus what code must enforce deterministically, actor/tenant isolation, retrieval-time filtering, DB-vector consistency and reconciliation, deletion/retention semantics, operational observability, and schema migration. The author treats concrete artifacts in Honcho — work_unit_key, sync_state, a reconciler process — as investigative evidence, while explicitly noting that full lifecycle and runtime verification remains future work.

For engineers, the value isn't endorsing one tool but having a repeatable checklist of state transitions, failure boundaries, and provenance chains to interrogate before adopting any AI memory OSS in production.