When AI Reviewers Disagree: Building a Multi-Agent DevSecOps Tribunal with Qwen-Max

Short summary

The author built a multi-agent DevSecOps tribunal where two AI reviewers negotiate disagreements using a deterministic cost-budget system rather than LLM-generated confidence scores. The LLM only picks categorical positions (DEFEND, PARTIAL, CONCEDE) while Python handles all math, making verdicts auditable and reproducible. The system scored 95% on a 40-case benchmark versus 82.5% for single-agent baseline, with Bayesian-smoothed track records persisting across PRs.

• •Multi-agent tribunal uses deterministic cost budgets instead of LLM self-reported confidence
• •LLM picks categorical positions; Python computes all math for auditability and reproducibility
• •95% benchmark accuracy vs 82.5% single-agent baseline, with persistent agent track records

Generated with AI, which can make mistakes.