alt Hacker NewsIn 2017 I worked tirelessly with my colleagues to implement and replicate the first transformer paper.
Yesterday I left Opus 4.8 to go do some architecture research, with GPU access.
It replicated and trained a credible baseline. It implemented some ideas I'd been thinking about, and wrote custom CUDA kernels for them. It read and summarised dozens of related papers.
It has since run dozens of experiments, with minimal supervision. When a model is unstable it kills it, documents why, fires off a new configuration.
The realisation that frontier labs are doing this at scale with unlimited GPU and token budgets.
It actually scares me a bit. The realisation that the next big breakthroughs will only have light human involvement.
The prospect of recursive self improvement feels more to real to me all of sudden
Replies
I've been doing the same. take papers, define a high level goal, then let it iterate. I have access to DGX boxes and watching the model rewrite stuff to take NVLink into account after it discovered it was great :-)
Yeah i find it very ignorant, at the current state to assume exponential growth etc. is all fantasy and everything is just hype.
I think its more like driving very fast, keeping an eye very close to the road and not knowing if there is a speed limit ahead very soon or not.
I'm assuming you saw this from yesterday:
https://www.anthropic.com/institute/recursive-self-improveme...
We are at the foot of a very sharp upward trajectory.
This is an interesting read: https://ai-2027.com/
I'm not going to say it's a perfect prediction, but I do find the trajectory of "can write something reasonable" to "oh can write snippets of code" towards larger and larger systems feels like it's played out - the common thing I see more now is that people talk of "taste" that the humans are contributing more than the raw coding part.
I get what you mean with this rather automated research, I've done it on a smaller scale with performance work because it can run/test/measure/propose changes/debug and loop. I can throw a vague idea at it, guide it or discuss with it and go and make a coffee.