The original title is a question: "How robust are natural language autoencoders to initialization?"

Short summary

Natural language autoencoders claim to interpret LLM activations as plain text explanations. This study reveals NLAs maintain reconstruction accuracy even with implausible initializations, and training decreases plausibility in well-initialized models—questioning whether NLAs reliably capture actual model reasoning.

• •NLAs trained from implausible initializations achieve near-identical reconstruction accuracy while outputting 99.3% implausible statements
• •Well-initialized NLAs decrease in plausibility during training (21% → 7.6%), contradicting expectations
• •Findings suggest NLAs may not be reliable for genuine model interpretation without further research

Generated with AI, which can make mistakes.