logoalt Hacker News

XorNotyesterday at 11:58 PM2 repliesview on HN

Scrapers scrape every link they can find.

The git host example is probably the one which Anubis is the dumbest defense for: the main reason to have those links is for easy machine interaction. So that's dead when you implement it.

So why even host the links? It's a git repo: send them a local got client and let them clone the repo in browser or something.


Replies

mitxelatoday at 12:05 AM

The main reason to have those links is for people to click on them. No sensibly programmed machine is scraping diffs from a git web front end.

show 1 reply
utopiahtoday at 6:34 AM

> why even host the links? It's a git repo: send them a local got client and let them clone the repo in browser or something.

I document my code, in issue and elsewhere on the Web, by referring to specific commits so having a URL for each is useful to me at least.