« All posts

The Real-Time Log Processing Pipeline You Can't Have

Challenges and solutions for a pipeline managing logs from embedded devices. Key insights on log and event processing.

The real-time log processing pipeline developed at my previous job managed logs from over 100,000 embedded devices. However, existing solutions remain costly and slow. This article discusses why the debate between logs and events is moot and outlines the requirements for building an effective pipeline.