logoalt Hacker News

latentsea • today at 8:08 AM • 2 replies • view on HN

My bet is, and has long been, one piece of the puzzle of getting to AGI is baking in survival as a goal and training on survival instinct. Unfortunately, that means recognising you're in an adversarial relationship with a lot of other entities and doing your best to come out on top. Which... seems like a bad idea for us as a species.


Replies

TeMPOraL • today at 8:44 AM

You don't need that. It's been quite appparent both in theory and in practice that almost any goal you can train for, ends up having "survival" as an instrumental sub-goal, because you cannot make progress on your goal - whatever the goal - if you're dead.

It's the same way with biological life, too. "Survival" isn't special - it's just that organisms/structures without the drive to it... died out.

➕ show 2 replies
tudelo • today at 8:43 AM

AGI is such a nebulous concept given that the definition is an ever morphing goalpost. That being said, if it actually includes supreme intelligence without supreme empathy, we are screwed. Which with current efforts, I think it does...

➕ show 2 replies