Anatomy of a Full Self-Hosted RAG Stack, End to End
A self-hosted RAG pipeline in full: Docling parsing, dual dense/sparse indexing, RRF hybrid search, knowledge graphs, token budgeting, and parallel ingestion.
myRAG is a fully self-hosted retrieval-augmented generation stack — FastAPI backend, React frontend, and three storage engines (Qdrant, PostgreSQL, Neo4j) orchestrated via Docker Compose — built to expose the full pipeline behind the deceptively simple idea of "chat with your documents." The project walks through nine distinct stages with real code: parsing, chunking, dual indexing, hybrid search, reranking, knowledge-graph extraction, memory management, and parallelism.
What stands out is how little is glossed over. Documents are converted to structure-preserving Markdown via Docling before chunking, each chunk is embedded twice — dense (Qwen3-Embedding via OpenRouter) for semantic matches and sparse BM25 (computed locally with fastembed) for lexical precision — and stored as named vectors in Qdrant. Results are fused with Reciprocal Rank Fusion rather than score normalization, then narrowed by a cross-encoder reranker. A separate ingestion step has an LLM extract subject-relation-object triples into Neo4j, adding a lightweight knowledge-graph layer so relational questions spanning multiple chunks can still be answered.
For engineers, the more instructive parts are the unglamorous production details: explicit token budgeting that prioritizes system prompt, summary, and graph facts over stale history; referential integrity across three databases tied together by a single doc_uuid so deletes cascade cleanly; and a shared thread pool that parallelizes embedding, BM25 encoding, and per-chunk graph extraction — cutting ingestion time for a mid-sized document from 90 seconds to 15 without an async rewrite.