logoalt Hacker News

achronotoday at 1:57 AM0 repliesview on HN

Certainly, real-life is the ultimate benchmark. But for various reasons that isn't always immediately possible to go evaluate a model on.

Maybe my Greek idea sounded too high falutin' or simply seemingly clever (I give an example below -- try it out!).

So here's something way simpler that Qwen3.8 27B does not get; only GLM-5.2 and K3 do.

**

Analyze the 2 structural (not semantic) patterns in this text:

1. Morning revient.

2. Birds saluent Morgenlicht.

3. We suivons Waldwege toward maison.

4. Rain tombe plötzlich; we cherchons Schutz beneath sapins.

5. Night vient langsam; we trouvons Wärme near le Feuer, sharing quelques Geschichten together.

**

The answer should get not just the obvious cyclic E-F-G pattern but also the word counts being Fibonacci. Surprisingly few models get this. The only way I got Qwen to do this was on the 2.4T model, with extensive prompt scaffolding. Claude (Opus & Fable), Sol, Grok etc. got it on the first attempt. (All models, all attempts max reasoning level.)