alt Hacker News> At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.
Is there a distinction in your mind between consciousness and intelligence? Is it possible, for example, for a machine to solve complex problems but not be conscious? Or vice versa, can an animal or a person be very unintelligent yet still conscious?
Replies
A philosophical zombie [1] is essentially a mobile, autonomous human body devoid of consciousness. Critically, still giving all of the external cues that it has in fact consciousness. It is supposed to prove that physicalism doesn't work. "The sense of consciousness" is used like a "soul, with extra steps".
In my humble opinion, which I have no way to prove or disprove, consciousness ("as a soul with extra steps") does not exist, and we are all philosophical zombies. Consciousness "as an amalgamation of complex biological signals and neural interactions that has evolved through millions of years as a successful survival strategy" does exist, and that is all that is needed.
The question is mainly interesting in the context of our own behavior and interactions. Because would we treat our food differently if we decided e.g. pigs were intelligent and conscious? I had some salami on my pizza yesterday. But I also believe pigs are intelligent and conscious. And they are close relatives of us genetically. Some people choose to not eat meat for this reason.
I'm of the school of thought that we are all biological computers with emergent properties like intelligence, consciousness, etc. that we might eventually succeed in replicating. Maybe at a more modest level at first. From a practical point of view, I'm more interested in intelligent agents than conscious ones. I mainly need them to do useful things for me. Too much consciousness is a double edged sword. Because then I would have to consider how my agent feels about all this. But can you really have one without the other? I don't have a good answer to this.
People have a tendency to anthropomorphize everything around them. Definitely their pets, plants, and in some cases even inanimate objects. Which doesn't help the debate because it's already happening with LLM tools. Even though they are probably still on the definitely not conscious side of the fence even when they demonstrate mildly intelligent reasoning occasionally.
At some point, people will have a hard time telling the difference with AGI. Is it all smoke and mirrors at that point or are they dealing with an intelligent/conscious thing? That's no doubt going to entertain philosophers for centuries to come. But from a practical point of view, does it really matter at that point? If we can't reliably tell the difference, is there still a difference?