alt Hacker NewsThe real elephant in the room is pricing for KV cache writes and reads. That makes all the difference for tasks with large context.
Replies
ianberdin • last Monday at 7:33 PM
[dead]
The real elephant in the room is pricing for KV cache writes and reads. That makes all the difference for tasks with large context.
[dead]
Cache writes/reads are the majority of the picture for agentic development. When you see these people saying a coding run "used 200 million tokens" or whatever, most of that is cache reads, so it should be the headline price IMO (and is one reason why DeepSeek API is so striking with its minuscule cache pricing).