Research · 2026-11-03

Chinese AI models fall further behind OpenAI and Anthropic once tests go private

Independent firm Artificial Analysis found that after switching to secret tests, the gap between leading American models and Chinese open-weight models grew from 3 to 12 points.

✨ What happened

Artificial Analysis, a company that tests AI models, published new results. Chinese models with open weights (which can be downloaded and run independently) turned out to be significantly weaker than the leading American ones. After the testing methodology changed, the gap became four times larger.

📌 Why the results changed

Previously, the test tasks were known in advance. Developers could train their models specifically for them, so the tests showed more than the models could actually do. Now the tasks are secret. Under the old system, the best open models trailed the leader by 3 points; under the new one, the difference is 12 points.

🔹 Examples with numbers

Moonshot's Chinese model Kimi K3.1 Pro scored 32 points in its category. Anthropic's flagship Claude Opus 5.5 scored 58 points. Even the smaller Claude Haiku 5.5 outperformed every open alternative.

🔸 What experts say

Snowflake CEO Sridhar Ramaswamy asked whether open models are really catching up or whether the test may have leaked. Part of the market has long said that rankings can be tuned to tests, so such numbers should be read carefully.

💬 Why it matters to ordinary readers

The claim that cheap Chinese models will soon displace ChatGPT and Claude looks less convincing. According to OpenCode co-founder Dax Raad, DeepSeek's daily traffic could be served by a single Nvidia server costing $6-7 million, while ChatGPT has more than 1 billion monthly users. The scale difference is still very large.

Sources: 🔗 DOU/dev.ua