alt Hacker NewsYea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma.
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
> If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.