Optimizing RAG at Scale: Chunking, Hybrid Retrieval, and Reranking for 95% Recall

Short summary

A detailed engineering walkthrough of rebuilding a RAG retrieval layer for production, covering four chunking strategies (fixed, recursive, semantic, agentic) tuned per document type with measured recall@10 metrics. Hybrid retrieval combines vector search and BM25 via Reciprocal Rank Fusion, then cross-encoder reranking funnels 50 candidates to 5 for a 15% recall gain at 50ms cost. Query transformation expands user questions into multiple search queries to improve retrieval quality.

• •Four chunking strategies tuned per document type achieve 91-97% recall@10
• •Hybrid retrieval fuses vector + BM25 via RRF, then cross-encoder reranks 50→5
• •Cross-encoder rerank costs 50ms but gains 15% recall over bi-encoder alone

Generated with AI, which can make mistakes.