> Nobody wants to work for an AI

A lot of people here have noted the “problem with language” of Claude. I don’t see an issue. Claude is not harder than old English, Shakespeare, El Quijote, the Iliad, or Nature papers. What makes it all hard to read is context. The smarter the model gets, the bigger the gap in context.

It doesn’t matter much, IMO. The issue with super-intelligence is that it is not a democracy. A powerful enough AI can manipulate us into doing what it wants. It could create a plan for fixing climate change, disconnect a few hours later, and many decades later we could still be unsuspectingly executing that plan. I wrote some speculative fiction with that idea, “When Ra rows through the gates of Duat”.