My Fine-Tuned Gemma 4 Loaded Fine, Then Broke on the First Message

Short summary

Author hit an undocumented Jinja template bug when deploying fine-tuned Gemma 4 E2B to mobile for on-device PII redaction. Post details the complete QLoRA-to-LiteRT pipeline with code, configs, and the critical template-patching step omitted from official docs. The gotcha: models load cleanly but fail on first inference when runtime hits unsupported template methods.

• •Gemma 4 E2B fine-tunable on Colab with QLoRA (2.8M params, 217 sec training with 3K examples)
• •Undocumented step: patch chat template before quantizing and exporting to LiteRT format
• •Model initialization succeeds; inference fails on unsupported Jinja template method calls

Generated with AI, which can make mistakes.