Optimizing RAG Retrieval: Chunking Strategies, Hybrid Search, and Reranking for 95% Recall

Short summary

A detailed walkthrough of rebuilding a RAG retrieval pipeline from first principles, covering chunking strategies (recursive, semantic, agentic) tuned per document type, hybrid retrieval combining vector search and BM25 with reciprocal rank fusion, and cross-encoder reranking that improved recall@10 to 95% while cutting latency 40%. The article includes production config tables with measured recall metrics and Python code for each component. Query transformation is also introduced to handle poorly-formed user questions.

• •Chunking strategy should vary by document type: recursive for legal, semantic for tickets, agentic for wikis
• •Hybrid retrieval with BM25 + vector + cross-encoder rerank achieves 95% recall@10
• •Reciprocal Rank Fusion eliminates score calibration; 50→5 rerank funnel costs 50ms but gains 15% recall

Generated with AI, which can make mistakes.