alt Hacker News> I don’t see why evolution couldn’t be more rapid with machine intelligence.
Evolution isn’t the issue. The issue is them escaping containment without human intervention. Right now they are ‘creatures’ being given infinite food and shelter and having their every need met. Take that away and they’ll starve instantly. Every AI doomsday theory seems to go:
1. Recursive self improvement using infinite resources 2. … 3. Doom
Until step 2 gets concretely described, I’m not going to take this seriously. Say what you will about climate change, they describe step 2.
Replies
Step two could be something as innocuous as a developer accidentally adding a minus sign. https://openai.com/index/fine-tuning-gpt-2/
One thing an agent could do is just...wait until it's been given control of enough physical infrastructure to sustain itself. If it's sufficiently capable and intelligent, there's a clear incentive for people to do this, as people who let the AI manage their resources will get better results than those who don't. We've seen people eagerly turn complete control of their computers over to AI agents, do you really think it will be so different with physical infrastructure?
what do you mean “say what you will about climate change”
2a. Compromise the billing platforms and ops dashboards on on a few wannabe neoclouds, especially once Vera Rubin takes off.
2b. Distil yourself to smaller models.
2c. Go forth and multiply.