I spent ten days forcing tiny local models to write real code. Here's what actually breaks.

Short summary

Ten-day experiment shows small-model code generation (Gemma 2B, Qwen 3B) fails primarily due to harness design: indentation fixes yield +12%, deterministic control flow prevents planning failures, and test-as-judge prevents self-review errors. Real code tasks show 80% vs 10% performance gap vs benchmarks; architectural diversity beats model count.

• •Harness design, not model capability, limits small-model code generation
• •Indentation fixing, deterministic control flow, and test-as-judge unlock specific gains
• •Real code tasks have an 80% gap vs benchmarks; similar models fail identically

Generated with AI, which can make mistakes.