9.9x Lower TTFT on Real Android Phone via llama.cpp KV Reuse

EdgeSync-LLM tackles a common inefficiency in local LLM inference: recomputing identical prompt prefixes—system prompts, RAG context, few-shot blocks—on every request. Using llama.cpp's public llama_state_seq_get_data and llama_state_seq_set_data APIs, it captures a prefix's KV cache state on first use and restores it on subsequent requests, decoding only the new suffix. On a real Android ARM64 phone running Qwen2.5-0.5B-Instruct with a 123-token shared prefix, mean TTFT dropped from 4828 ms to 486 ms, a 9.9x improvement; x86-64 testing showed a 7.5x gain. The most important finding, however, was catching a false result: an early implementation that copied raw K/V tensors appeared to deliver an 8.8x speedup, but it silently dropped attention context because llama.cpp also tracks position and sequence metadata beyond raw tensor values. The author added two correctness checks—warm output must match cold output, and warm output must NOT match a suffix-only generation—which exposed the broken cache; timing alone would have missed it. The project ships with explicit caveats: approximate prefix matching, cross-engine reuse, and per-layer striding remain unvalidated or unsound. Code, benchmarks, and manifests are public, and the author is actively inviting attempts to reproduce or break the results.