Signed Symmetric Quantization for Few-Bit Integers

Short summary

This paper introduces signed symmetric quantization, a third option between standard symmetric and asymmetric quantization for few-bit integers. The signed absmax grid places the extra negative representable value on the dominant-outlier tail without a zero point, retaining the runtime profile of symmetric quantization. Validated on Qwen3, Qwen3.5, and Llama3 models, the method improves perplexity and few-shot accuracy over standard symmetric quantizers at no extra inference cost.

• •Signed symmetric quantization avoids clipping positive outliers without asymmetric runtime penalty
• •Method is conditionally bound-optimal on L2 error for 88-99% of weight groups in pretrained LLMs
• •Improves perplexity and few-shot accuracy on Qwen3, Qwen3.5, and Llama3 at no extra cost

Generated with AI, which can make mistakes.