ZVec's optimize() Can Silently Erase Crash-Recovered Vector Data

ZVec, Alibaba's embedded, in-process vector database (PyPI package zvec, v0.5.1), passed a basic kill-9 durability test: after 551 documents were inserted and the process was killed with os._exit(1), reopening the collection replayed the write-ahead log automatically and restored all 551 documents intact. The index_completeness: 0.0 value returned by stats() looked alarming at first, but testing across multiple clean, never-crashed collections showed it's simply the default state for any unbuilt index — queries against it still return correct results.

The real danger surfaces when optimize() is called to build that index. On a collection that has been crash-recovered via WAL replay, optimize() silently drops the recovered documents during its index-rebuild path, with no exception, warning, or flag in the return value. The loss is only visible by reopening the collection from a separate process or fetching individual document IDs. Crucially, the trigger isn't unflushed data — a clean-shutdown collection with unflushed documents survives optimize() fine. The determining factor is whether the collection has ever crashed and gone through WAL recovery.

For teams embedding ZVec inside orchestration layers like Temporal Activities for durable state, this changes how recovery logic must be written: index_completeness: 0.0 is not a corruption signal, and calling optimize() on a crash-recovered collection is precisely the operation that can turn fully recovered data into partial or total loss. The tested mitigation is to fetch and reinsert recovered documents into a fresh, crash-free collection before ever invoking optimize().