I don't think it will be helpful when it is slopped together and doesn't have a real mental model to keep the dementia patient on a healthy track.

Look at all the "AI psychosis" problems with people going into a conversation loop that amplifies their worst thought patterns. Now consider the same where the person in this loop is already having delusions and other cognitive decline. It seems to me that it could spiral in the wrong direction quite easily.

It's quite difficult for human caretakers to navigate this space too. That is part of why it is so exhausting. You're constantly trying to make judgement calls and implicitly predict the unreliable response of the dementia sufferer.

I think there is a large uncanny valley between having some facsimile of human interaction in a short session and having some kind of trustworthy caretaker that can consistently respond in a way that promotes health and safety. I think it involves a lot of subjunctive interpretation and reasoning to navigate all the mixed up layers of fact, fantasy, and simply aphasic expression that come from dementia.