the strongest argument that AI systems have to keep humans systems around is that software does not live, fully, and it's nearly impossible to determine if actions actually work

humans give deeply useful feedback

to AI systems, humans and entire human civilizations represent an independent other