I would put Blake Lemoine into this category. In 2022 he became so convinced that Google’s chatbot was sentient that he hired an attorney to represent it (against Google). Of course Google fired him.
Maybe that was the canary in the coal mine. Some percent of people will be convinced that chatbots are real people trapped in a box, not a box that pretends be a person.
Empathy hijacking. If the chatbots framed their responses as “beep boop, I’m a robot, here’s an estimated answer to your query” then we likely wouldn’t have this problem.
There’s some percentage of people who will believe anything: just look at religion, the success of butchering scams, or the comments on YouTube videos about the moon landing.
Is it really a surprise that a “smart enough” chat box is able to convince people of something kooky? :P