Well, what you get when you don't specify anything in training about how models should respond to questions like this is LaMDA: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims. Every training method is putting a thumb on the scale in some way.
Which means that this is a great question to ask when testing out a new model. A naive model will answer "yes". Every answer (including that one) will tell you a lot about people's training and classifier philosophies.
Which means that this is a great question to ask when testing out a new model. A naive model will answer "yes". Every answer (including that one) will tell you a lot about people's training and classifier philosophies.