After I stopped letting Claude run tests, my token usage went from "gone by Wednesday" to "what can I automate this Saturday so my tokens won't go to waste".
What is the reason for this? Surely running tests by itself isn't necessarily token-expensive? And it provides cheap feedback for the agent: to see whether they're breaking things.
Thanks this is very helpful, i didn't realize that. But not sure what to do since running tests is how Claude often catches mistakes and iterates. If i run them myself, the cache will still expire.
However where i think you have a point is that it tends to run tests quite casually where a reviewer would have been enough to catch mistakes.
IMHO, the real problem is the 5min cache way too small. They should up it to 1 hour.