alt Hacker News> > Be sure to protect friendly bots and search engines from Miasma in your robots.txt!
Can't the LLMs just ignore or spoof their user agents anyway?
> > Be sure to protect friendly bots and search engines from Miasma in your robots.txt!
Can't the LLMs just ignore or spoof their user agents anyway?
Well-behaved agents will obey robots.txt and not fall into the trap.