logoalt Hacker News

weknowbetter • yesterday at 8:05 PM • 4 replies • view on HN

You say this as if recursive self improvement is inevitable.


Replies

Beached • yesterday at 11:06 PM

I think it is. Do you think it isnt possible?

GodelNumbering • yesterday at 8:37 PM

All proponents of RSI seem to massively underestimate the problem of verification. Verifying that a hypothetical version N+1 is better than version N is an extremely hard problem. Especially so because you only have two possible verifiers - humans and the version N - both of which are out of their depth wrt N+1's capabilities. Not saying this is impossible, just that it is hard and slows things down.

➕ show 6 replies
atleastoptimal • yesterday at 10:34 PM

It may not be inevitable but there are less reasons imo to think it is not inevitable than inevitable.

In favor of inevitable

1. LLM's reaching human level at any task within 2 years once they show sparks of competence in

2. Trillions of dollars in capital going directly to recursive self improvement and AGI as the holy grail of AI development, with this being the explicit goal of every frontier AI company

3. Reliable improvements in coding and mathematics with every model release, with no signs of slowing down

4. Thousands of metrics to evaluate model ability, allowing automation of many facets of pre-training and post-training models without need for human judgement at every point

Not in favor of inevitable

1. Fear of what may happen once it is reached leading to apprehension from the labs

2. The "last mile" problem of AI research ends up making it essentially infeasible to scale models to that point

FL33TW00D • yesterday at 8:31 PM

We've seen one singularity before!