alt Hacker NewsEven ignoring superior caching on a local setup, Mac hardware can often process input token around 10x as quickly as they produce output tokens. Openrouter seems to have only a 2x difference on the same models.
Even ignoring superior caching on a local setup, Mac hardware can often process input token around 10x as quickly as they produce output tokens. Openrouter seems to have only a 2x difference on the same models.
For larger contexts (eg. 20,000+ token agent workflows), being 10x faster still isn't enough. You have to be close to ~100x faster at crunching contexts for it to feel like realtime.