logoalt Hacker News

salomonk_mur • today at 4:54 PM • 2 replies • view on HN

I'm surprised at the effectiveness of simple PoW to stop practically all activity.

I'll implement Anubis at low difficulty for all my projects and leave a decent llms.txt referenced in my sitemap and robots.txt so LLMs can still get relevant data for my site while.keeping bad bots out. I'm getting thousands of requests from China that have really increased costs, glad it seems the fix is rather easy.

Replies

gruez • today at 5:02 PM

>I'm surprised at the effectiveness of simple PoW to stop practically all activity.

It's even dumber than that, because by default anubis whitelists the curl user agent.

    curl -H "User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36" "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/?id=v7.0-rc5&id2=v7.0-rc4&dt=2"
    <!doctype html><html lang="en"><head><title>Making sure you&#39;re not a bot!</title><link rel="stylesheet"
vs

    curl "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/?id=v7.0-rc5&id2=v7.0-rc4&dt=2"
    <!DOCTYPE html>
    <html lang='en'>
    <head>
    <title>kernel/git/torvalds/linux.git - Linux kernel source tree</title>
➕ show 2 replies
wolvoleo • today at 5:13 PM

It's definitely more than enough to stop me as a human wanting to visit the site, so yeah.

In that case a better solution would be to take the site down altogether.

➕ show 1 reply