Back to feed
arXiv cs.CL
arXiv cs.CL
7/14/2026
Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences

Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences

Short summary

This study benchmarks faithfulness of LLM-generated clinical trial summaries across GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash for healthcare providers, patients, and payers. Unsupported claims were the dominant failure mode across all models. A knowledge-graph-augmented RAG system produced statistically significant faithfulness improvements (p<0.0001), with model-dependent improvement pathways.

  • •200 clinical trials evaluated across three LLMs using a six-dimension faithfulness schema
  • •Unsupported Claims identified as dominant failure mode (mean 1.55/3) across all models
  • •Knowledge-graph-augmented RAG significantly improved faithfulness scores (entailment +0.0125, p<0.0001)

Generated with AI, which can make mistakes.

#research-breakthrough#ai-tools
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering
arXiv cs.CL
The original title is a survey title, not a news headline. I need to rewrite it as a news headline for a mobile feed.
arXiv CS.AI
The original title is about RAG evaluation with RAGAs metrics. Let me rewrite it to be punchy and specific while preserving key facts.
Dev.to
Large Language Models as Unified Multimodal Learners for Clinical Prediction
arXiv cs.CL