ReportQA: QA-Based Radiology Report Evaluation

Short summary

ReportQA introduces a QA-based evaluation framework for automated radiology report generation, using LLMs as judge models to score reports against clinical knowledge. The resulting QAScore metric shows stronger alignment with radiologist judgments than existing NLG metrics. The authors release code, knowledge trees, and datasets, demonstrating that question-driven inference outperforms traditional report-based approaches.

• •Novel QA-based metric (QAScore) for radiology report evaluation with superior radiologist alignment
• •Uses LLM-as-judge paradigm guided by clinician knowledge trees and structured information extraction
• •Open-source framework with code, datasets, and pipeline for reproducibility and extensibility

Generated with AI, which can make mistakes.