Towards surfacing model algorithms with meta-tokens in the J-Space

Short summary

Researchers applied the J-lens interpretability technique to Qwen3.6-27B and discovered 'meta-tokens' — single tokens that reveal non-obvious internal computations, such as Chinese tokens firing when the model detects ambiguity or hedges. Steering these meta-tokens causally changes model outputs, e.g., suppressing a hedging token makes the model commit to a single answer. The work is a proof of concept that J-lens can surface algorithms, not just intermediate variables, with multi-token J-lens as the natural next step.

• •J-lens applied to Qwen3.6-27B surfaces 'meta-tokens' revealing internal model algorithms
• •Causal steering of meta-tokens changes model outputs (e.g., suppressing hedging makes model commit)
• •Chinese tokens are more information-dense, making them richer meta-token candidates in J-lens readouts

Generated with AI, which can make mistakes.