Building Service Topology at Scale: Architecture and Challenges

Netflix engineers aim to create a real-time service dependency map to troubleshoot issues more efficiently. The initial version functioned well in a local environment, but faced challenges in production, including slow Kafka consumers and memory issues. This post discusses architectural decisions for scalability, production challenges encountered, and lessons learned throughout the process.