It's not just comparing all the models, it's also comparing all the providers and configurations of those models.

If you're doing any kind of production AI work you'll end up with outages caused by calling a single provider, OpenRouter seamlessly switching between providers is a godsend for uptime.

But even more than that there's meaningful cost+speed differences.

Here's Sonnet 4.6 being served direct, via Amazon and via Google

https://la9q13gg8w.evvl.io/

(spoiler: Google was both fastest and cheapest)