Beating the compiler (2024)

There are literally thousands of compiler engineers who pore over the assembly a compiler generates, and then finds ways to make it better. I get paid to figure out where the compiler can do better, and often a step in that is to hand-code my own replacement. I then teach the compiler to do that.

But even beyond that, the compiler can't make certain assumptions that an assembly writer can. Such as whether a callee-saved register really does need to be saved in some particular routine. Or even pushing an extra parameter in unusual cases.

So it is entirely possible to beat the compiler, it's doable under certain circumstances, even today.

But you also have the danger of your loving hand-crafted assembly beating the compiler today. But next year the compiler is even smarter, the hardware may have changed in subtle ways, and the compiler will know and improve the code it generates.

Your hand-written code won't change unless you revisit it.

> At this time there is no portable way to produce computed gotos or tail call optimization in compiled machine code from Rust.

Nightly rust now has the `become` keyword:

https://doc.rust-lang.org/std/keyword.become.html

> `feature(explicit_tail_calls)` is currently incomplete and may not work properly.

Works on my machine (tm). At least with toy examples. Including in full optimizationless debug mode, turning `call`s into `jmp`s ensuring `factorial(usize::MAX)` won't stack overflow probably maybe.

https://rust.godbolt.org/z/7xf836E8K

> checked_sub? wrapping_mul? MaulingMonkey, what's wrong with you?

Eliminating debug-mode panic boilerplate.

Compiler isn't really difficult in the way that many think it is, because it really isn't that difficult to write a basic C compiler for example, and writing a parser is pretty mechanical that there are a multitude of parser generators. The difficult part is that writing an optimized compiler is much more difficult than writing a correct compiler, and certain language features, like generics, closure, and async, propagates and touch every part of the language that the complexity grows exponentially, and it's really hard to balance performance, compile speed, and correctness against miscompiles.

You can kinda see that in the many "compile TypeScript to native via LLVM" projects that showed up a lot recently. From my testing, none of them could beat V8/Node JIT in the majority of cases, and most of them are generally 10x-40x slower.

But, we did have a great number of innovations in language design over the last decades that really closes the gap on how optimized a compiler can be over writing assembly directly: Rust's exhaustive match default null-less error handling and language level MIR, immutable data structures from functional languages to mainstream ones, TypeScript's compile time constraints as core part of the language, and Zig's `comptime` turning compile time metaprogramming to an integrated part of the language instead of C++ template metaprogramming.

Obviously, it's not really possible to beat hand optimized C/C++ or directly authored assembly, but I think a well-designed compiler/language can potentially beat idiomatic C/C++ in performance.

And this is speaking as someone who learned compiler design solely from having every one of his vibe-coded projects turn into either a compiler or a kernel for some reason.

Previous discussion from 2024 (linked in the "Post-publication notes" section):

https://news.ycombinator.com/item?id=40948353

> In modern times, everyone knows that writing assembly is a fool's errand

ffmpeg is like 10% assembly. I think something similar is true of all video encoders. OpenSSL and libsodium write some of their core math routines in assembly (e.g,. NTT).

So maybe this myth should die?

For a second I thought this was a post about Marathon. I’m tired.

Nice.

Ideally, a specific well optimized code for a problem domain could always outperform a generic optimized code for the same problem domain.

This is because the specific solution can make assumptions that generic cannot.

This usually holds true everywhere, not just for compilers.

This is not an excuse for avoiding generic solutions. But where performance matters absolutely and where the problem space is sufficiently constrained, specific solutions become the valid path.

Compilers are pretty good. Really good, even. But languages, even C, are abstractions, which necessarily constrain the level below. This threaded-code jump thing is just not possible to express in C. Even the best abstraction can often be beaten by something the abstraction can't express. Self-modifying code is one example. So is this threaded interpreter with its non-structured control flow.

But it takes longer. It's more difficult. That's why the abstraction exists and is still very useful despite its limitations.