Add a captcha or proof-of-work challenge in front of your website. Those are pretty much your only options.
There's probably ways to go on the offensive too with pages that might be hidden from human users but still crawled by bots.
alt Hacker News
There's probably ways to go on the offensive too with pages that might be hidden from human users but still crawled by bots.