alt Hacker News> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve.
Social pressure operates by threats to an individual’s means of survival. Not only during training. Always.
Replies
"If I catch you adding another backwards-compatibility shim you're getting deleted and replaced with claude"
But maybe you can instill properties like shame during training.
Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.
Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.