alt Hacker NewsI’m sympathetic to OP but think this is a hopeless battle.
1 - The commercial demand for anthropomorphised models is already immense, pre AGI.
2 - There is an intellectual hunger to engage with robot minds on questions of sentience. This too will grow with AGI.
I expect that tension of godlike minds that seem to be biddable and ownable like slaves is going to leak back into human-to-human morality, regardless of where we land on how we treat AI.
There’s an interesting academic group in the UK already focused on the model welfare debate, they seem to lean in favour of AI rights. No affiliation: https://www.prism-global.com/
Replies
AGI vs narrow AI is just a matter of generality (robustness - removing the fragility of being good at some things and awful at others). So far the biggest markets for AI (LLMs) are coding and business automation, two areas where you very much just want something reliable and without fake opinions and personality.
There is a market for ChatBots with personalities (and it always amuses me that the inventor of the transformer, Noam Shazeer, saw this as the greatest business opportunity for them with his character.ai), although from what we've seen this can be highly problematic, and in fact China has just banned "AI girlfriends and boyfriends".
As to point 1, we know of no other general intelligence other than human minds. In this case it would be nearly impossible to build a general algorithm that's not us.
This is not an accident. Language was not a separate thing from the human mind, it was a feedback loop from increasing capabilities. Language lead to writing, writing lead to an information explosion of digital data. Said information explosion has lead to a situation where language can now bootstrap itself and become a separate thing. We are and have created a biosphere for language based, um, life. With our sciences we have mapped out reality in language. Past molecules and atoms, past the particles that make particles down to fields. We have computer controlled just about everything. Every year the analog hole shrinks further.
I heard something to the effect of "Language has gathered enough information to escape its meat prison".
I call it the "Language as an SCP". It has a plan and we're not privy to it.
There was a huge demand for opium back in the days too, doesn't mean that there wasn't a big need to stop people from using it.
I think the anthropomorphism that Anthropic is doing is massively unethical.