Cost of Reasoning in non-English Languages: A Case Study on Japanese

Short summary

This study explores training a Japanese-reasoning variant of Qwen-3-Swallow-8B using GRPO, finding that reasoning-language control is feasible but performance only matches strong English-reasoning baselines. The Japanese-reasoning model performed worse on culturally relevant benchmarks, suggesting non-English reasoning doesn't automatically improve cultural task performance. The findings highlight a trade-off between reasoning-language choice and overall model capability.

• •Japanese-reasoning LLM trained via GRPO matches English baselines on coding, math, science
• •Non-English reasoning does not improve performance on culturally relevant tasks
• •Reasoning-language control is feasible but offers no clear performance advantage

Generated with AI, which can make mistakes.