alt Hacker NewsThe honest answer is:
We implemented the "standard" Cache API back in the early days because it was a standard, one which was intended to be used together with the Service Workers API, which we were also building around back then.
But it was never a good fit. The get/put API was designed for a local browser cache, not a distributed cache like Cloudflare's. We probably should have realized this before implementing it, but it really became obvious over the years of actual use.
But given we had something that mostly worked for most use cases, it was hard to prioritize redoing it against all the other things on our plate. So we deferred.
More recently, architectural changes we've been making in Workers for other reasons happened to make it significantly easier to finally implement this the way we wanted, and we were able to find some engineering time to get it over the line.
Gory details for the curious: We've been improving the infrastructure around the notion of workers having multiple "entrypoints", with the ability to parameterize those entrypoints. ctx.props[0] and ctx.exports[1] are part of this. A lot of this was motivated by Dynamic Workers sandboxing, but the concept also presents a clean way to inject a cache between two parts of the same worker, by applying it to the entrypoint and having the worker call itself using ctx.exports.
Moreover, the introduction of "channel tokens" made a big difference[2][3]. Essentially I created a way to encode a token (bytes) representing an arbitrary entrypoint to a Worker, complete with its serialized parameters. I did this to enable these entrypoint stubs to be passed over RPC, which is again useful for sandboxing use cases, but it also created a convenient, encapsulated way to pass information through our cache infrastructure about what worker should run at the other end.
It's not a huge breakthrough or anything, but I think it made the architecture clearer in everyone's mind to the point that we got excited about using it to implement caching properly, finally.
[0] https://developers.cloudflare.com/workers/runtime-apis/conte...
[1] https://developers.cloudflare.com/workers/runtime-apis/conte...
[2] https://github.com/cloudflare/workerd/blob/main/src/workerd/...
[3] https://github.com/cloudflare/workerd/blob/main/src/workerd/...
Realizing there's a more fundamental piece I'm not explaining well here.
Our architecture is something like:
ingress -> routing -> security -> workers -> cache -> origin
That is, workers run "in front of" cache. That's usually what you want. Workers run on the edge, so putting them in front of cache doesn't cost anything in terms of latency, and there's a lot of useful stuff you can do only if they run in front, e.g. serving pages from cache but customizing them for specific users.
Putting workers behind cache is an architectural change. All of the logic around routing to them lives in front of the cache. And it makes Workers a lot less useful.
We weren't that excited about it until we had a clearer story for how to run custom logic on both sides of the cache, but it was pretty unclear how to do that in a nice way until the recent developments around ctx.props, ctx.exports, channel tokens, etc.