> And it gets churned by every single request they receive

It gets pretty efficiently cached, but does eat the context window and RAM.