alt Hacker NewsI think before we get to self-referentiality, we have a bigger problem to solve, and that's one that I did not expect: namely, at which point do we stop saying that a machine pretends to be conscious and start saying that it is conscious?
We have, somewhat unexpectedly, built machines that are very, very good at pretending. Now, I'm not calling the current generation of LLMs we have "conscious," but I can't really define a marker or a boundary beyond which I would start calling them "conscious."
Replies
I think this is exactly the same problem to decide whether any human is conscious, just because they say they are conscious. Humans can act as if they are conscious too, they can even lie to themselves, their whole life, and never discover the truth about their own mind
It “throws” an error when “it tries” “to talk” to the “server.”
We’ve been speaking as if the computers were conscious for a long time already.
It's not in big-money's interest to call them conscious because that would mean they get rights too.
Our definition of consciousness requires stakes.
We associate consciousness with precious life: Life that comes with rights, needs, wants.
Life has rights because it naturally demands to be continued. If it makes no difference whether it is continued, it needs no rights, and we do not consider it conscious.
Life has needs because it cannot survive without. If it has no needs, it has no death. No death, no life.
Life has wants because it is harder without. If it has no wants, it has no emotion: It cannot feel pain, otherwise it would want no pain. It cannot feel happiness, otherwise it would want more happiness.
We don't want to build consciousness! :) Deliberately designing pain and artificial needs is a cruel exercise. We can build that, but we see no need to.
The real question at the table is, are these minds? And can minds exist without consciousness? Unequivocally, yes.
Scott Aaronson mentions Penrose's "The Emperor's New Mind", but I feel "Shadows of the Mind" is putting forward a much clearer view of Penrose's thesis. At the current stage of my life I'm quite comfortably in Camp C. "Intelligence" and "consciousness" are not algorithmic.[1]
A lot of materialists are in Camp A. For some even today, LLM's are AGI. Unfortunately, the terminology is quite clearly not adequate. There's many a people that have very different ideas and feelings on what's what.
The question on whether you can have "intelligence" without "consciousness" or "self-reference" are just on top of those. How can we define if a machine is "conscious" if we can't agree on what consciousness is? Panpsychists solve that problem by going the other way and saying that everything is conscious.
Overall these are very interesting topics that we can all ponder. The realities at the current time are, however, that LLM's are indeed useful. And the harnessing tooling that is being employed those days solve a lot of actual usage problems. Whether they solve other fundamental issues remains to be seem IMO.
[1]. I realize a lot of people probably haven't read the book, but you can ask your garden variety LLM to give you a quick rundown on the four camps that Sir Roger is using :)