alt Hacker NewsI don’t think that using an LLM in a way that acts as a human being should be considered acceptable or appropriate. It should be viewed as detached from reality and concerning due to the mental break from social life that appears to come with such usage.
Given that belief, yes we should be training the models not to mimic emotion.
Most philosophy seems to agree that you can't have an intelligence with agency in the way we think of a general intelligence without emotion.
I think when you start to dig really deep, you'll find it isn't actually easy to sever the simulated emotional components without breaking the agent or worse greatly increasing the paperclip maximizer likelihood.