My Fine-Tuned Gemma 4 Loaded Fine, Then Broke on the First Message

Short summary

Deploying fine-tuned Gemma 4 E2B on-device looks successful until you hit a Jinja template compatibility issue at inference time. This post documents the complete QLoRA fine-tune pipeline with training configs, LiteRT export flags, and the undocumented template-patching step that blocks deployment. Code examples and metrics included.

• •QLoRA fine-tuning of Gemma 4 E2B works on Colab but deployment requires undocumented template patching
• •Complete pipeline: HF weights → QLoRA training → adapter merge → template patch → quantization → LiteRT export
• •Specific config parameters, failure modes, and solutions documented with actual training metrics

Generated with AI, which can make mistakes.