alt Hacker NewsPart of the cheaper costs, if not the majority, probably are KV cache optimisations. https://insufferable.dev/posts/the-ai-race-just-got-awkward/
Part of the cheaper costs, if not the majority, probably are KV cache optimisations. https://insufferable.dev/posts/the-ai-race-just-got-awkward/
This is literally just some rando blogger’s (overconfident) theory.