logoalt Hacker News

ConfusedDogtoday at 3:34 PM2 repliesview on HN

Why would 3.6 flash perform a little worse than 3.5 flash on Artificial Analysis Coding Index...

https://artificialanalysis.ai/models/gemini-3-6-flash?intell...


Replies

sosodevtoday at 3:38 PM

Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.

show 2 replies
wmedranotoday at 4:09 PM

Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?

artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.