it’s quite suspect to claim that a model is ‘underperforming’ so resoundingly, especially after so little time since release.

there’s this objectivist approach of reducing llm quality to a single dimension, furthermore one with measurability bias.

my thesis is that the chinese models beat the american models on benchmarks, on quantitative measures, but probably do worse in subjective measures. to put an example, compare a chinese phone with an iphone, the specs are great, but the OS? the materials? the privacy posture?

so will chinese llms have great specs and benchmarks, but will miss the mark when it comes to the actual quality in practice when your input stops looking like the benchmarks it was trained on.

America number one, it’s the chinese that play catch up with clones, not the other way around