logoalt Hacker News

icepush • yesterday at 5:50 AM • 1 reply • view on HN

Step two could be something as innocuous as a developer accidentally adding a minus sign. https://openai.com/index/fine-tuning-gpt-2/


Replies

jquery • yesterday at 3:41 PM

A misaligned model is only one small part of step 2. Now this misaligned model has to suddenly acquire more power than every single other AI on the planet. It has to be immune to shutdown, manufacturer its own replacement hardware, and acquire chips, energy, raw materials, etc., with vigorous human opposition (this is an extinction scenario that AI doomers are predicting, after all)

Nobody has satisfactorily explained step 2 other than “well, it’s a superintelligence” which sounds lot to me like “it’s God”.