logoalt Hacker News

calvinmorrisonyesterday at 9:35 PM1 replyview on HN

cache all of them? no... but you could cache some of them, likely the newer are accessed more frequently.


Replies

zbentleytoday at 2:41 AM

Per the article, the vast majority of traffic is scrapers that are generating effectively random, permuted URLs. Even if they're already caching their popular-with-real-humans most recent data, that efficiency gain is a tiny drop in a very large bucket.