» Tag
edge-ai
11 posts9.9x Lower TTFT on Real Android Phone via llama.cpp KV Reuse
EdgeSync-LLM reuses llama.cpp's public KV APIs to skip reprocessing shared prefixes, cutting Android TTFT 9.9x while catching a fake 8.8x speedup.
56M-parameter LLM runs distributed across three ESP32-S3 boards via ESP-NOW
A 56M-parameter LLM was split across three ESP32-S3 boards over ESP-NOW, using Split-PLE quantization and a KV cache for coherent generation.
28.9M Parameter LLM Runs Fully On-Device on an $8 Microcontroller
An 28.9M parameter LLM runs entirely offline on an $8 ESP32-S3 chip using Google's Per-Layer Embeddings technique for flash-based storage.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comRK3576 NPU on Mainline Linux: Single-Task Works, Multi-Task Stalls
Mainline RK3576 NPU driver bring-up fixes an int8 requantization bug but hits a hard wall on multi-task jobs; full root-cause ledger published.
Inside Android's 16.67ms Race for Real-Time 60 FPS Segmentation
How Android engineers hit 60 FPS real-time video segmentation within a 16.67ms frame budget using NPU/GPU acceleration, AICore, and quantization.
A deterministic memory layer for local LLMs at near-zero cost
An open-source deterministic memory layer bolts onto frozen LLMs, enabling zero-forgetting recall beyond context at near-zero marginal cost.
Mastering Edge AI: Building High-Speed Vision Analyzers on Android
Explore the balance between deep learning and mobile device constraints. Learn about the architecture and Kotlin patterns for high-speed vision analyzers.
Mastering Custom C++ Kernels for Edge AI on Android via NDK
A technical look at bypassing Kotlin's abstraction tax for on-device LLMs using JNI, zero-copy buffers, and NPU-aligned C++ kernels via the Android NDK.
Benchmarking Edge LLM State Collapse: Drift Gauntlet & CSMS
Drift Gauntlet benchmarks recursive LLM state collapse on Jetson Orin Nano edge hardware, while CSMS runtime governance prevents it with near-zero latency cost.
No cloud, no GPUs: Liquid AI's LFM2.5-2.6B model empowers edge devices
Liquid AI introduces LFM2.5-2.6B, a model that runs on local hardware without cloud or GPU dependencies.