I spent ten days forcing tiny local models to write real code. Here's what actually breaks.

Short summary

Tiny local models (2B–3B params) can generate code competently with optimized harnesses, but hit a real wall on complex multi-step changes (~11% on real repo commits vs ~80% on isolated functions). Model routing by problem class outperforms stacking; test gates matter more than self-reflection. Cost and sovereignty emerge as critical constraints.

• •Small models succeed with harness fixes: deterministic control flow, test-based gates, re-indentation on parse failures
• •Real capability wall: ~11% on repository commits vs ~80% on single functions—rarely published honestly due to benchmark contamination
• •Model stacking fails unless models have uncorrelated failure modes; sovereignty and marginal inference cost dominate as subsidies end

Generated with AI, which can make mistakes.