When they answer back to us in personal pronouns, we will always be wondering if it is like LLM just putting most probable words together or something really sentient.

When someone makes a virtual girlfriend of it, is it really a disembodied person or just a smart answering machine?

A whole lot of ethical and psychological issues are to open up here.