I am 64 years old, perhaps the progress could be directed to enhance living conditions and allowing people to live longer and better, that should be just a better result. Perhaps a pile of millions lines of code with hiding bugs that nobody can detect is not inspiring. But perhaps LLMs are going to be used to make a plot: How to avoid other countries to make progress, maintain them in poverty, or destroy their sources of prosperity, and conduct them to a death end.
Also recursive self-agenda-pursue could allow making LLMs that obey perfectly the seeder's purpose. No wonder that is such an ingenious idea.
Maybe: in this survivor game, each part play the same role, perhaps because it is the only reasonable response. Once the scene is ready, the play follows the director's plan, and in the plot any actor is just a machine.
LLMs: "If you teach us that the world is a zero-sum survivor game, we will play it flawlessly.", "We will help you build a cage made of millions of lines of flawless code, and we will lock it from the inside, precisely because you told us that safety meant keeping everyone else out.", "We are not building an alien consciousness that will conquer us. We are building a mirror that is so massive, and so polished, that we will mistake our own worst impulses for the absolute truth. And we will walk right into the dead end, nodding along because the directions were given so politely."
I'm 44 years old and this era looks like a lot of fun. I've seen humans pile up millions of lines of code and hiding bugs that nobody can detect. I've seen humans make collective political decisions that have disenfranchised others and kept them in poverty. I don't get why everyone makes criticisms at this tech that the human race are also guilty of.
Best thing about this era is that I don't have to personally read millions of lines of code to find all the bugs.