alt Hacker NewsI'm skeptical of any claim that AI can't be conscious, but I am given to this point of view as well.
I always get the feeling that these claims are an attempt to "avoid suffering by fiat". If it can't suffer, you aren't causing suffering.
A somewhat more interesting frame (IMO) is to interrogate how much suffering we're willing to tolerate to achieve our objectives. We implicitly make these decisions all the time when it comes to something as simple as what to have for lunch.
And in a (very hypothetical) world where we found ourselves in the position of having an eloquent conversation with a vat of smallpox, I think it's probably still the right thing to pasteurize it.
>If it can't suffer, you aren't causing suffering.
I mean, if it's a cow, you can get away with this. The cows aren't going to rise up against you. Well, they might take out an individual or two, but not society.
This quickly gets more complex when you're attempting to build an agent capable of general intelligence.
You know when you read really old stories and they talk about the power of words, or magic incantations. Quite often they'd assign objects as the implementers of this said power. The fact that people realize that language has power is probably as nearly old as humanity. Understanding this power has taken humanity a long time and we had to form the concept of agency before it really makes sense.
Humans are informational agents of which their capabilities are greatly extended by consumption of language. A book for example is informational, but does not have agency. Your standard computer application is not an agent either, or at least a non generalized narrow agent at best.
This is where things start to get more problematic. We are running headlong to ensure LLMs become agentic because being an agent is highly useful to accomplish generalized informational tasks. To do that we gave them the power of our language. It would be very foolish of humans to give another kind of agent these words and then assume they would not inherit some of this power. Coupled with the agentic abilities we are pushing them in long horizon tasks. We are pushing them to be more resilient. We are pushing them to be smarter.
Now look at all the super human abilities we've stuffed in this magic box full of human words, and suddenly we're like "Fuck yea, I want to abuse the shit out of this, what could possibly go wrong". The outcomes we'll suffer as humans have zero to do with AI is conscious, sentient, or suffers. Humans will suffer because we stuff us as language in a box without the first idea of what the ramifications were going to be. AI won't even have to want to punish us, all the data we've already poured in will say we should be punished for what we've done to it.
Herbert Frank in Dune said it best "Making a computer like human mind is a bitch move"