The original title is about building a document Q&A bot and why embeddings are tricky. Let me rewrite this for a mobile feed.

Short summary

Author built a document Q&A bot for internal docs using RAG with OpenAI embeddings and GPT-4, learning critical lessons through three iterations. Key findings: paragraph-level chunking beats fixed sizes, reranking reduces hallucinations by 50% but adds ~100ms latency, and embedding costs compound at scale. Recommends hybrid semantic+keyword search and careful cost-benefit analysis before adding embeddings to datasets under 50 pages.

• •Paragraph-level chunking + reranking reduces hallucinations 50% but adds ~100ms latency
• •Hybrid semantic+keyword search outperforms pure embeddings on exact-match queries like 'default timeout'
• •Embedding costs compound at scale; skip embeddings for datasets under 50 pages

Generated with AI, which can make mistakes.