alt Hacker NewsCompilers can’t really, in a meaningful way, change the layout of your data in memory. And you do need to think about your memory layout to get any benefit from SIMD. You’ll notice a lot of compiler auto vectorization insert many instructions just to shuffle data around to get to a usable layout, which negates much of the benefit.
Replies
This is actually a very nice question and the answer is that interpreted languages with a JIT benefit from this.
One example is Java, which will happily vectorize your code into AVX or SSE where possible.
Python just got a JIT compiler and we’ll start seeing the same thing soon.
But as someone else said here, some constructs don’t translate well and adding transformations to show vectorization would negate the perfomance gains.
Sad that the compiler (even Java) can’t explain you this and warn about it, but now with LLM, maybe they’ll start doing things like that soon.
Depends on the programming language. A good question is why we don't have more optimizable languages in mainstream use.
Even when the layout is friendly to simd, auto vectorization can be finicky. As a programmer, it’s really annoying to be constantly inspecting compiler output to see if the code was properly vectorized. Even if it was, slight changes or compiler updates can throw the whole thing off. Auto vectorization is nice when you get performance improvements for “free”, but I find it fragile for the really critical parts where you absolutely need it to be vectorized.
I often wonder about a macro-like thing where we could write a function using a subset of the language that’s simd aware. A bit higher level than using intrinsics or those simd libs