logoalt Hacker News

Tharretoday at 2:39 AM0 repliesview on HN

Those would have the user agent "ChatGPT-User" though, and I barely see those. The majority comes from "GPTBot" like in my excerpt above, which makes it pretty clear that it's used for some sort of training:

"GPTBot is used to make our generative AI foundation models more useful and safe. It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models."