alt Hacker NewsI am actively opposed to this design for a first compiler. There is no need for a lexer with a recursive descent parser. Register allocation is also an unnecessary distraction. It is better for a first compiler to compile to a higher level language in which neither register assignment nor memory management are necessary.
Optimizing compilers are suboptimal in that they waste enormous amount of time optimizing code that can't or needn't be optimized and even where the optimizations are helpful, they are opaque and at risk of unexpectedly regressing both due to small changes at the source code level or changes in the compiler optimizer, both of which are quite insidious.
If instead of optimizing compilers, we had languages that allowed for seamless interop between low level and high level functions, then perhaps an llm becomes the optimizer (or you can invoke the compiler to optimize a specific function by source level rewrite). The benefit of this compared to a traditional optimizing compiler is that the optimization is done once per function and never repeated (until prompted) and the implementation is human readable and not buried in a binary. Moreover, and perhaps even more importantly, by not doing optimizations in the compiler, compilation times can be much faster, easily 100-1000x than state of the art optimizing compiler, while generating equivalent or even better runtime performance. As it has been said: premature optimization is the root of all evil.
I disagree with several parts here. But hopefully, this leads to a fun discussion!
> no need for a lexer with a recursive descent parser
I'd argue that teaching how to write a lexer + recursive descent parser is more relevant in the context of production compilers: many major production compilers out there use hand-written recursive descent parsers (cpp, javac, rust, go,javascript...). Recursive descent parsers are also really nice for emitting error messages.
> It is better for a first compiler to compile to a higher level language in which neither register assignment nor memory management are necessary.
Compiling to a high-level target can be a reasonable first project(e.g., you can emit LLVM), but imo its a different objective from learning the full stack. Emitting actual ISA instructions(even sub-optimally, after all it's a university course) forces you to learn calling conventions, isel, register pressure, stack layouts etc. Building a compiler,at least for me, is probably one of the easiest ways to understand how all of it works together.
> optimize a specific function by source level rewrite
I don't think replacing optimizations with a per-function source-level rewrite works as a general model. Many optimizations are not local to a single function (for example, inlining function calls can lead to new constant-propagation opportunities). If your argument rests on the fact that not all functions are hot, a lot of general-purpose JIT compilers out there already use runtime info to decide when to optimize hot functions, so part of what you're proposing already exists.
> implementation is human readable and not buried in a binary
Is this really a requirement for your program? In most cases, I think the optimization story is more like: "code you want to write" != "code you want to run"
> Moreover, and perhaps even more importantly, by not doing optimizations in the compiler, compilation times can be much faster, easily 100-1000x than state of the art optimizing compiler, while generating equivalent or even better runtime performance
I think the actual answer here is "it depends". For long-running programs, one tradeoff is build time vs future execution time. Also many optimizations cannot be expressed in source code itself. For example, in C++, you can do stuff like whole program de-virtualization only at link time, which is why LTO exists.
Aside: I personally work on source-to-source automatic differentiation inside compilers, and I can give examples for missed optimizations in generated derivative code if you don't run existing optimization passes like LICM/CSE before differentiating a function.