logoalt Hacker News

100mstoday at 7:21 AM2 repliesview on HN

> Google's strange job optimisation technique (for jobs running on hard disk storage)

Can you give more context on this? Opus couldn't figure out a reference for it


Replies

why_only_15today at 7:40 AM

This is a quite old technique. The idea, as I understood it, was that lots of data at Google was stored in triplicate for reliability purposes. Instead of fetching one, you fetched all three and then took the one that arrived first. Then you sent UDP packets cancelling the other two. For something like search where you're issuing hundreds of requests that have to resolve in a few hundred milliseconds, this substantially cut down on tail latency.

show 2 replies