Building a Multimodal Vector Search Platform for Product Catalogs
How a recommendation team built a multimodal vector search platform with Qdrant, tuned HNSW settings, and optimized GPU usage for embedding millions of products.
A recommendation engineering team built a vector search platform that encodes millions of product images and descriptions into multimodal embeddings, enabling meaning-based search rather than keyword matching. This approach solves both the mismatch between differently-worded but visually identical products and the cold-start problem for new items with no behavioral data. The architecture is deliberately simple: an embedding service, an ingestion pipeline, a stateless search API, and a config service, backed by Qdrant as the vector database.
A key insight was treating multitenancy as a data modeling decision: instead of one collection per tenant, the team used shared collections with per-tenant index graphs, improving both latency and isolation as a structural property rather than a filtering afterthought. Another major finding was that manually tuning HNSW's ef parameter made searches 3-5x slower than leaving it on automatic mode, reinforcing the principle of always benchmarking before overriding defaults.
On the GPU side, utilization mattered far more than raw speed. Scale-to-zero spot GPU pools paired with a producer/consumer pipeline cut costs significantly, while a silent fallback to full precision on GPUs lacking BF16 support caused unexplained slowdowns with no error messages. The team also stressed that the least glamorous parts of the system, like building resilient image-downloading pipelines, ended up being the hardest and most critical engineering work.