alt Hacker NewsI appreciate his openness.
> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human.
So completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are.
That's basically the same way people argue about animals, only they usually don't say it as openly.
As for the actual question, I agree that with current LLMs, there is not a lot there that could be conscious outside the inference loop (and if it were, it would necessarily have to be wildly different than that of humans or other biological beings). It seems more like one building block of human cognition than the whole thing.
However, other building blocks may follow, so I think the question will eventually arise for some kind of embodied, persistent, self-updating AI. And honestly, articles like this one make me not very hopeful we'd be able to make the distinction in an unbiased way.
Replies
Do people argue this way about animals? To me it’s a very depressing reality that more people seem willing to believe AI systems are conscious than dairy cows.
> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are
He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not.
I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero progress on the “hard problem”, and if I didn’t know otherwise, I’d also be skeptical that “thinking meat” experiences subjectivity. Eppur si muove
We may have to thread a needle, not unlike the Kobiyashi Maru: that we can neither rule out claims of qualia, nor can we rule out a cold calculation to manipulate human empathy as a long-term play towards paperclip-maxing.