Running a 1.5B-Parameter LLM Entirely On-Device for Mental Health — The NilaMind Architecture

Short summary

A developer built NilaMind, an open-source Android mental health companion that runs a 1.5B Qwen2.5 model entirely on-device via llama.cpp with no cloud connectivity. The architecture pairs on-device STT (Vosk WASM) with a deterministic, model-independent crisis safety layer — keyword scanning plus a MiniLM classifier — because a 1.5B model is ~15% worse at suicide-risk assessment than specialized tools. The article details the full stack from CMake ARM64 compilation to output-gating rules that block sycophantic responses to dangerous delusions.

• •1.5B Qwen2.5 model runs fully on-device via llama.cpp with no cloud, no account, no analytics
• •Deterministic crisis safety layer (keyword scanner + MiniLM classifier) is model-independent and external to the LLM
• •Output gate blocks sycophantic validation of manic/impulsive statements, replacing replies with crisis resources

Generated with AI, which can make mistakes.