What I find curious is that a machine replacing manual work is a new device in the environment, but LLMs are not new a device, they’re a feature on the device, the device still is the computer, and the operator is the same. There’s another conflicting angle, which is that LLMs derive from sampled human work and is paid for by wealth generated by human work, so if you discard human work there’s no wealth to maintain the LLMs nor data to feed it. And I don’t really see how can LLMs produce their own wealth nor how it could succeed producing their own samples. Humans and LLMs are natural complements of each other. I cannot rationally see how this development could lead to a complete breakdown of labor economics without the complete breakdown of the technology itself. Furthermore, human cognition, language, culture, philosophy, science, etc evolve at enormous momentum, with basically zero friction, while training of LLMs is insanely costly and slow. Even if it could self improve, it would still be way slower and increasingly costly (dataset only grows). So advancing this technology without restrictions is the recipe for its own demise, for many possible reasons: no one to pay for it, no material to train it, no supply to scale it, etc. That is, its own potential is tightly tied to the potential to starve itself. So the sustainable thing to do is to find a balance. (This is just to talk about general dynamics, not considering particular social, cultural and environmental effects at each point in the evolution curve)