Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment

Short summary

This paper evaluates nine open-weight language models (135M–3B parameters) on a structured benchmark for local deployment, finding that Qwen Coder 3B leads at 75.67% strict accuracy. A shared parameter-efficient fine-tuning pipeline using 4-bit NF4 quantization with DoRA/LoRA adapters on an NVIDIA L4-class budget improves performance significantly—Qwen Coder 3B gains +26.85 points. The authors conclude that disciplined benchmarking and low-cost specialization already make sub-3B models viable as local experts for structured niche workloads.

• •Nine open-weight models (135M–3B) benchmarked on 1,085 examples across 16 topics for structured local deployment
• •Parameter-efficient fine-tuning with 4-bit NF4 quantization on L4-class budget yields up to +26.85 point improvements
• •Sub-3B models are already viable as local experts for structured niche workloads with proper specialization

Generated with AI, which can make mistakes.