I don't quite agree with the Assembly => Compilers => LLMs as some sort of "stages of evolution" argument tbh.

LLMs are an additional way to generate code, but it's not like code generation or "low-code" tools are a new thing that made compilers and programming languages obsolete.

LLMs "just" allow to provide the code generation input directly in human language. But whether providing such a specification in "sloppy" human language instead of a precise specification language is actually a good thing for long term projects is still arguable (as convenient as it is).

Mathematicians and engineers invented their own precise "DSLs" (e.g. math notation and blueprints) for exactly this reason, and high-level programming languages are nothing else than "specification languages" that don't leave room for the ambiguities of "natural languages", and it's still not clear to me how having both a natural language specification and a precise specification in form of source code side by side can be considered a good thing. It's easy to say of course that the "natural language specification" is the single source of truth and the source code is a throwaway output artifact, but then you're back at having a specification written in a language that's useless for expressing precise intent.

(and this is also the fundamental difference between compilers and LLMs and why I believe that comparing LLMs to compilers is wrong: determinism. The output artifacts of a compiler are literally throw away. Nobody in their right mind would commit object files into version control, while nobody in their right mind would not commit the source files generated by LLMs into version control, because creating the source files from scratch would yield different results on each run).

Releasing the harness as part of LLM-generated projects still makes sense of course :)