IIRC there was a blog post here on HN recently explaining in detail why Anubis doesn't keep out AI bots.