WiFi Weight Streaming for LLM on ESP32
Run Llama-2 architecture on ESP32 via WiFi streaming. RAM usage remains flat, while generation is limited by network bandwidth.
A method to run a Llama-2 architecture model on an ESP32, where weights are stored on a PC and streamed over WiFi. This approach maintains flat RAM usage regardless of model size, limiting generation by network bandwidth instead.