I Benchmarked Chinese vs US AI Models: The Numbers Don't Lie

Short summary

Author benchmarked 8 AI models (4 US, 4 Chinese) over 14 days and found Chinese models cost 9-60× less while matching or exceeding US flagships on MMLU and code benchmarks. Performance gaps are minimal (3.5 MMLU points max); on HumanEval, DeepSeek V4 Flash matched GPT-4o within margin of error. Accessibility barriers—payment methods, documentation, phone verification—remain the primary obstacle, not technical capability.

• •60× price spread between Claude 3.5 Sonnet ($15/M) and DeepSeek V4 Flash ($0.25/M) for comparable performance
• •Chinese models within 3.5 points on MMLU and <1.5 points on HumanEval despite massive cost difference
• •Integration barriers (payment, documentation) matter more than raw capability for Western adoption

Generated with AI, which can make mistakes.