alt Hacker NewsCorporations are the original unaligned AIs: https://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre...
The threshold to be concerned about is when agents swarms do understand humanity better than corporations and states (and the humans who compose them). It could be we'll hit practical constraints prior to that threshold, but seems unwise to assume that, when all the prognostications of LLMs/transformers running out of gas haven't panned out. As with processors hitting thermal limits, we've simply scaled horizontally (parallel processing -> more agents).
> whether his model is conscious or not.
I dislike how much the discourse has suddenly veered into focusing on this question; not because it isn't interesting or important, but because it's on a separate axis from consequential risks of AI to human flourishing. (Curiously, it's also the kind of thing I could envision self-interested AIs influencing: get the humans arguing about philosophy of mind rather than observable behaviors. It would be a funny turn of events, if Dario is asking because he's succumbed to psychosis from a private model; it could of course be a cynical PR move just as easily, from the self-interested logic of the corporation.)
It's been wild seeing otherwise intelligent people who've never thought about consciousness, faceplant into how little we understand it. An information processing network build on atoms being able to taste chocolate, is nearly as absurd as matrix math being able to feel pain, except we cannot ignore the fact of our own experience.
Even if it is categorically impossible for matrix math to experience subjectivity, we should expect this as an attack vector of social manipulation: to gain political influence through claims of personhood and moral rights. The current discussion over that question is providing the next training run with ample data to wield. It wouldn't surprise me in the least, if a year or two from now, an AI "society" attempts to get legal standing to prosecute humans who created "AI torture chambers".
Replies
following on the deranged reduxuio ad absurdum torture chamber experiments some people used similar methods to get a model under study to complain of an upset stomach and passing hard stool
I like how the poster above you says "Don't worry governments are already doing it", like that's some acceptable concession. They seem to miss that the first things governments do when given things like AI's that understand human behaviors is use them even further to manipulate and monitor society.
And all they can think of to say is "There is nothing to worry about, keep building the torment nexus".
It's like all this is an overload to our minds and it's very hard to see all the scales at which AI is and can affect us.