Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination

Short summary

Research on four open-source medical LLMs shows hallucinations are readily detected in neural activations (AUROC 0.77–0.86) but cannot be reliably controlled through neuron-level steering. The detection signal is distributed across many neurons, meaning targeted manipulation fails despite the underlying structure being visible. Findings indicate hallucination mitigation requires fundamentally different approaches beyond neuron identification.

• •Hallucinations detectable but not controllable via neuron steering in medical LLMs
• •Detection signal is widely distributed and redundant, not localized to specific neurons
• •New approaches needed beyond neuron-level intervention for hallucination mitigation

Generated with AI, which can make mistakes.