Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

Short summary

A detailed engineering walkthrough of rebuilding a RAG retrieval layer for production, covering chunking strategies by document type, hybrid retrieval combining vector search and BM25 with reciprocal rank fusion, cross-encoder reranking, and query transformation. The approach achieved 95% recall@10 and cut latency by 40%. Includes production config tables and Python code for each component.

• •Document-type-specific chunking: recursive for legal, semantic for tickets, agentic for wikis
• •Hybrid retrieval with RRF fusion and cross-encoder reranking improves recall 15% over pure vector search
• •Query transformation via LLM expands user queries before retrieval, improving match quality

Generated with AI, which can make mistakes.