Hybrid Multi-Tenant Architecture for Stateful AWS Services

A large-scale ad-serving platform generating billions in annual revenue had outgrown its cellular AWS architecture, which assigned a dedicated account, VPC and ALB to every tenant. Supporting just 18 clients across four regions already required 181 separate targets, servers sat idle 98 percent of the time, onboarding a new client took 52 days, and the setup couldn't handle concurrent high-traffic events. Because the platform keeps tenant data in memory for performance, sharing clusters also created a noisy-neighbor problem where one tenant's large dataset could trigger out-of-memory issues for others.

To fix this, engineers built a hybrid multi-tenant architecture organized around three nested levels: tiers group tenants sharing a common footprint, cells represent account-level scale-out units, and infra groups are self-contained units bundling a VPC, ALB, per-tenant ECS clusters, IAM roles and monitoring. Downstream connectivity via AWS PrivateLink is now pre-wired once at tier creation instead of per tenant, so newly onboarded tenants automatically inherit full connectivity.

Amazon Route 53 weighted routing lets traffic shift gradually across ALBs and accounts without any client-side DNS changes. Together, these choices cut network configuration overhead by 80 percent and turned tenant onboarding from a multi-week infrastructure build-out into a simple configuration change—offering a concrete reference model for engineers designing isolation-versus-efficiency trade-offs in stateful multi-tenant systems.