Building an Offline AI Note-Taking App with WebGPU

Short summary

The author built PrivateScribe, a browser-based AI note-taking app that runs LLM inference entirely via WebGPU with zero data leaving the device. Key advantages include zero installation, GPU hardware acceleration, and true offline capability. Major engineering challenges included managing browser memory limits for quantized models and implementing streaming token output to reduce perceived latency. The tool supports note summarization and email drafting/rewriting workflows.

• •PrivateScribe runs LLM inference entirely in-browser via WebGPU
• •Zero installation, hardware-accelerated, fully offline — no data leaves device
• •Key challenges: browser memory management for quantized models and latency mitigation via streaming tokens

Generated with AI, which can make mistakes.