alt Hacker News> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).
I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.
The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.
In general, people that behave similarly in many contexts can have quite different experiences in those contexts.
There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.
Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).
To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.
LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.
Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.
I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.
I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.
Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.
Yea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma.
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.