Building an LLM System from Scratch in Pure Python & NumPy: Architecture, Invariants, and Clean Code

Short summary

Draco AI is a fully open-source LLM system built entirely in pure Python and NumPy, eliminating dependencies on PyTorch or HuggingFace. It features ternary quantization (BitNet 1.58b) for 70-80% memory reduction, a 3-tier hierarchical KV cache, a 12-stage self-healing cognitive pipeline with 9 independent hallucination verifiers, and strict execution invariants for speculative decoding. The project includes a pytest suite and is available on GitHub.

• •Draco AI: open-source LLM system in pure Python/NumPy with zero AI-framework dependencies
• •Key features include ternary quantization, 3-tier KV cache, and 12-stage hallucination verification pipeline
• •Strict execution invariants ensure deterministic rollback and RoPE consistency across speculative decoding paths

Generated with AI, which can make mistakes.