How We Built Instant Translation Help (即时翻译帮助) with Python and LLMs

Short summary

LectuLibre built an instant translation help feature that shows contextual word explanations in under 500ms by combining a pre-built bilingual glossary with LLM fallback. The glossary is generated post-translation using spaCy for noun-phrase extraction and SentenceTransformers for cross-lingual alignment, stored in PostgreSQL with context sentences for disambiguation. At query time, a FastAPI endpoint checks the glossary first (with embedding-based context matching), then an in-memory TTL cache, and finally falls back to DeepSeek for rare terms—cutting latency from 3s to near-instant for most lookups.

• •Hybrid glossary + LLM cache achieves sub-500ms translation popups
• •spaCy noun-chunking and SentenceTransformers cosine similarity align source-target phrase pairs
• •Context-aware glossary matching with embedding similarity improves precision by ~30%, DeepSeek handles long-tail fallback

Generated with AI, which can make mistakes.