alt Hacker NewsIn case you are unfamiliar with Karpathy's Loop[1], it is a genetic algorithm[2] where the genetic "mutations" are clever-but-random ideas generated by an LLM agent, aimed at improving a system.
(1) Let the LLM randomly perturbate the system.
(2) Measure the system's performance.
(3a) If the perturbation improved performance, keep the change.
(3b) Otherwise, don't.
(4) Repeat
Replies
A genetic algorithm keeps a population, and there is a "crossing" operation.
I don't see both ingredients in Karpathy's proposed scheme.
i actually do it differently
> (1) Let the LLM randomly perturbate the system.
instead of this i ask LLM to what's least likely to improve performance and then measure it.
sometimes big gains come from places you thought are least likely.
Lol, I respect karpathy a lot, but this is such an obvious in your face idea that it is laughable to put someone’s name on it.
What’s next “karpathy investing” where ai in a loop builds a portfolio?
thanks, I thought as a researcher Kaparthy would include and cite relevant papers. I quickly became disappointed. I already knew openevolve and the ACE Framework paper. This is the first time I learned about Genetic Algorithm and I now have some clear roadmap for studying.
Wtf, this has a name now? I thought of this exact idea literally months ago but never had the time to do any experiments on it.
At the time I dismissed it as potentially being incredibly expensive for the improvement you do get, and runs into typical pitfalls of evolutionary algorithms (in the same way evolution doesn't let an organism grow a wheel, your LLM evolution algorithm will never come up with something that requires a far bigger leap than what you allow the LLM to perturb on a single step. Also the genetic algorithm will probably result in a vibecoded mess of short-sighted decisions just like evolution creates a spaghetti genome in real life.)
I'll definitely need to look into how people have improved the idea and whether it is practical now.
I was working some time ago on LLM assisted optimizations and algorithm discovery and this does not look like a novel idea.
AlphaEvolve from google is evolutionary algorithm which uses LLMs for Idea generation following very similar loop:
- https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...
- Open source implementation of the algorithm: https://github.com/algorithmicsuperintelligence/openevolve