alt Hacker NewsIt's hard to argue that it isn't an optimization, because it doesn't affect the semantics of the program. However most optimizations are very hard to observe. The vast majority of optimizations only affect code size and runtime. TCO is one of the few exceptions. It affects memory usage, and more sensitive stack memory at that. This is why a missed optimization can be so much more catastrophic and it is worth considering things like `musttail` attributes so that the code fails to compile rather than misses the optimization.
I can only think of a few other optimizations that affect memory usage. Register spilling (arguably not really an optimization but a necessity), Rust's niche filling for enum discriminants and C++'s std::vec<bool> (a language-level optimization, arguably a different thing entirely).
I often think about how few memory optimizations we have. The reason is most likely that they tend to be non-local so are much harder to apply than CPU optimizations that generally have no effect outside of the function they are in.
Replies
> It's hard to argue that it isn't an optimization, because it doesn't affect the semantics of the program.
Depends on the semantics of the programming language itself. For some languages, it is truly an optimization, for some, it is required, and does meaningfully change observed semantics.
If the semantics of 'while (true)' was "will crash the program after an implementation-defined but often fairly low number of iterations", I would stop using 'while (true)'.
std::vector<bool> is just a terrible specialisation, it isn't an optimisation.
If std::vector<bool> was an optimisation we couldn't write C++ which blows up because it's actually a bitset, it would be semantically transparent - but that's easy to do even by accident because it's not transparent at all.
In fact the existing std::vector<bool> should just be named std::growable_bitset or something and then std::vector<bool> would make what you actually wanted like Rust's Vec<bool> does.
> because it doesn't affect the semantics of the program
It does when you use them as a feature and not an optimization. Like in interpreters, state machines, parsers, etc.
Calling tail calls an optimization set computer science back 40 years.
If my program crashes without it, that's a semantic difference no?
JVM does a lot of escape analysis to turn heap allocated memory into stack local variables.
It doesn't matter if it's local since it's a VM, it's doing it at runtime and can change an entire call stack of non local code for an optimization.
> I can only think of a few other optimizations that affect memory usage
Java has string interning. I think that’s a hack that shouldn’t exist in an ideal world. Reason is that, as a library writer, you cannot make the call whether to intern strings (requiring more instructions for string access, thus slowing down code, but decreasing memory usage, and, because of that, possibly speeding up the code again) or not.
The main difference is not that it affects memory usage, imo.
It’s that it makes memory usage bounded when it’s on, and unbounded when it’s off.
In languages that have guaranteed tail call eliminations, the semantics of tail recursion is the same as that of a loop. So you can express the same iterative algorithm without using iterative code.
>It's hard to argue that it isn't an optimization, because it doesn't affect the semantics of the program
it is guaranteed in Scheme, and it affects the semantics of programs in a completely positive way.
Much of computer science is "pure" and "abstract" like mathematics. However, programmers are still taught to use loops to calculate factorial rather than recursion in order to avoid stack overflow. In Scheme you can use recursion without flinching. That is a semantic difference.
I think the problem with considering it a "pure optimization" is that code that is written to use tail-calls, if not optimized, is almost always unbounded recursive code. And modern OSes tend to have relatively small stack-size limits (relative to the kinds of huge data structures modern software slings around, incl. not only individually-"wide" structures, but also "deep" trees and graphs.)
Which means that "whether this naively-recursive code is actually recursive in practice" is a semantic difference, in that there is an error/failure-mode (stack overflow) that can be statically guaranteed to not happen (at least for a given compilation target) if TCO gets applied; but which cannot be guaranteed to not happen without TCO applied.
---
Tangent: you could of course try to write code defensively, to guarantee that a stack overflow won't occur, by bounding recursion separately (e.g. via a passed-and-decremented recursion-limit parameter), so that in the non-TCO case, you get a software exception thrown (which you'd hopefully then handle... somehow), rather than triggering a stack overflow.
And for many more-traditional recursive algorithms, this works!
But doing so for the types of algorithms that are "canonically" expressed in terms of tail-calls (even in a non-tail-call-idiomatic language like C), almost always requires poking holes in the C abstract machine to see through to the micro-architectural details underneath.
You can't just use something like a recursion-limit parameter as a general solution for these algorithms, as TCO is used in things like continuation-passing or threaded-code VM implementations — i.e. things that look less like visiting trees and more like visiting unboundedly-non-terminal infinite-state-machine states ["infinite" because the states are dynamic function pointers to JITted code, and more of them can appear at runtime.]
You need to not track the "number of invocations deep" you are into the algorithm, but rather, how big the stack actually is at the moment. Which means you need to actually do math on addresses of the stack base pointer vs either the stack pointer, or the address of a local stack-allocated variable. There's no version of that that doesn't require writing non-portable inline assembly.