My Release Gate Passed. The Model It Shipped Answered 'Neutral' To Everything.

Short summary

An ML engineer shipped a DeBERTa-v3 NLI judge model that passed all CI release gates but answered 'neutral' to every input after INT8 quantization. The root cause was per-tensor dynamic quantization destroying DeBERTa's disentangled attention — the release gate scored PyTorch weights, not the ONNX artifact users actually downloaded. Switching to per-channel quantization fixed the collapse, but quantized v3 still underperformed the smaller RoBERTa v2 model, making the bigger base a bad fit for quantized CPU deployment.

• •Release gate validated PyTorch weights but not the shipped ONNX INT8 artifact, hiding a total model collapse
• •Per-tensor quantization destroyed DeBERTa-v3's disentangled attention; per-channel scales fixed it
• •Quantized v3 still scored worse than quantized v2 (RoBERTa), so the bigger base was abandoned for CPU deployment

Generated with AI, which can make mistakes.