How /search and /ask Work: Local Hybrid RAG with ChromaDB + SQLite FTS5

Short summary

This article explains how Llamail combines semantic search (ChromaDB) with keyword search (SQLite FTS5) for hybrid email retrieval, solving the weakness of pure semantic search (missing exact matches) and keyword search (missing conceptual understanding). The hybrid approach normalizes scores from both retrieval methods and merges them, achieving ~3 seconds search and ~7 seconds full RAG answers on an 18K email mailbox. The implementation stores LLM-summarized email text in ChromaDB rather than raw content, keeping vector embeddings semantically dense while using FTS5 for exact term matching.

• •Hybrid semantic + keyword search overcomes limitations of each method alone
• •Normalizes and merges results with simple weighted scoring (simpler than RRF)
• •Implementation runs efficiently on consumer hardware: 3sec search, 7sec RAG Q&A

Generated with AI, which can make mistakes.