alt Hacker News> Fetched naively, that would be over 600 PB of git traffic per day.
This seems like an absurd amount of data / traffic per day. Is there really that much to code that needs changing on a daily basis?
Replies
No, of course not.
As the article explains, "over 600 PB" = 25M code checkouts * 25 GB "compressed working copy".
So "fetched naively" here means doing a full `git clone` on every need for `git checkout`, which of course nobody should do, nor does that refer to "code that needs changing daily".
I don't think anyone "needs" more than perhaps 640KB of git traffic per day, but we want more.
That 600 PB was mostly agents doing ”git clone” and downloading the entire history of their huge monorepo, as I understand it.
AI iterate quickly, with 100s of followups to fix results
No, that’s why they don’t do it that way.
This would be us doing a shallow checkout of individual commits (not cloning the full history) if we downloaded the 25GB compressed data for whatever commit we needed multiplied by the number of jobs that check out code.
We don't do it this way, for obvious reasons, but it means that there are different levels of caching and infrastructure required to make git work at repo sizes far smaller than what we have.