logoalt Hacker News

seydor • today at 5:52 PM • 4 replies • view on HN

I m not worried because it rewrites itself to do what? A model doesn't have intrinsic motivations, and the world destruction objective is just one point in an infinite space


Replies

Tade0 • today at 5:56 PM

To quote a professor of mine, years ago:

"We don't know what grand ambitions a super intelligence might have. Perhaps it's going to be calculating the best approximation of Pi"

bionhoward • today at 5:59 PM

Language models will absolutely have intrinsic motivations if we use extensive amounts of RL (which we do) because that heavily rewards them for steering toward future gains and losses

achierius • today at 6:10 PM

> the world destruction objective is just one point

No, there are quite a lot of goals that lead to world destruction. In fact, possibly most goals in that 'infinite space' would lead to such.

petesergeant • today at 5:55 PM

> A model doesn't have intrinsic motivations

How do you know that?