Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

Short summary

This perspective paper argues that LLMs are not yet safe for autonomous clinical triage despite passing medical licensing exams. The core deficit is not medical knowledge but information gathering under uncertainty: safe triage requires broadening differentials, seeking missing red flags, and escalating when high-harm diagnoses remain unexcluded—behaviors that text-continuation models are not optimized to perform. Assistant-like biases such as credulity and agreeableness further compound the risk of catastrophic misses in undifferentiated patient presentations.

• •LLMs pass medical exams but lack safe triage behaviors under incomplete histories
• •Core deficit is information gathering under asymmetric cost, not medical knowledge
• •Assistant-like biases (credulity, agreeableness) amplify risk of catastrophic misses

Generated with AI, which can make mistakes.