alt Hacker NewsYou are assuming that because you are giving the operator money that you are entitled to use their website however you please -- that's not how it works. If you enter a brick & mortar establishment and break their rules, even as a paying customer, you might risk e.g. being kicked out and banned, depending on the behavior. This is not unique to e-commerce
To be fair, I do not generally support wholesale banning of scrapers and non-browser clients, since this tends to have lots of ugly side effects.
But regardless, I'm interpreting your last comment as saying that you should be entitled to behave as you please because you are a paying customer. If that's the case, then you are in for a rude awakening one day
Replies
I think the implicit assumption is that if you explained to the owner of the website what the scraper was doing, they would be more than happy to allow it.
The bot was also being generally courteous, only scraping the 2-3 pages needed to get the information for the human who spun up the LLM.
I don't think GP was at all suggesting he should be able to act however he wants, just because he's a paying customer.
I run an ecommerce site and I'm actually more worried about bots being rejected than accepted because of stuff like GP. We get a decent amount of business through ChatGPT.
I think he's saying that his use of the site, via the LLM, is consistent with what the site owner-operator likely desires.
To give a concrete example: I am a physician and have a site for my private clinic. I am not only unopposed to being scraped by bots/LLMs, I prefer it. The only value I gain from users being on my site is referral to my practice, and that same information being surfaced by search engines and LLMs just means I'm getting in front of more potential patients.