alt Hacker NewsEveryone Should Know SIMD
Comments
I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns.
I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was like putting high-performance racing tires on a lemon with a broken engine.
It was the root-of-all-evil-type-premature-optimization, because I wasn't measuring performance, and I wasn't thinking about where the allocations were, etc. Now, I try to model my data as if it were SQL tables, see what my potential "primary keys" could be, and build my data structures around my access patterns.
For example, I used to model trees as structs pointing to other structs on the heap:
struct Tree {
tag: TreeTag,
children: Vec<&Tree>
}
But a tree can be represented a million ways, and can always be linearized. So now I really consider my access/insert patterns of the tree, whether it's really a tree or some other sort of graph, whether I can store it in a Vec or a Struct of Vecs, etc. Since really looking at things through their access patterns and "primary keys", my code has been much faster and simpler.
This has the added effect that a lot of your data ends up in homogeneous arrays / vecs, which means that the compiler can do its SIMD magic, the CPU can read it from your L1 cache a million times faster, etc. And then when you need to drop down into SIMD yourself, you can write some awesome branchless code.
1. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
> Every developer should… most importantly, not be scared of SIMD
Seems like he should be recommending fearless_simd [1], the Rust crate by Raph Levian and the folks at Linebender :)
More seriously, if you’re looking to add SIMD to your Rust code, that’s the package to start with.
Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqY
Tangentially for Go programming, the last time I looked at optimising some Go code with SIMD there were a few different options available, but they were either not maintained any more or had incomplete support and required first writing your function in C++ with intrinsics and generating assembly, then converting it to go assembly with a tool [1]. I never got my function to work in go despite the C++ code working fine. In short, not really a production ready option for Go. This was a year or two ago, though.
Edit, there's now an experimental official library at https://go.dev/pkg/simd/archsimd/ see https://go.dev/doc/go1.26#simd and at https://github.com/golang/go/issues/78902 so things have moved since I tried it last.
To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible.
Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the need), how to turn conditions and branches into masks, or simply things like "division does not exist" becomes a lot easier when you have spent at least some time trying to use SIMD yourself.
99% of developers should just ignore SIMD. Most projects have a lot of low hanging fruit to increase performance, and still nobody finds the time to solve them.
"More importantly, when this loop matters enough for me to care about a 5x speedup, I want the vectorization to be explicit and predictable. I don't want an unrelated code change or compiler update to quietly turn it back into a scalar loop."
Only tangentially related but this is by far the most painful part about optimizing code for JIT compilers like V8. Even changing a constant from 1 to 1.0 somewhere else can change the optimizations performed and lead to an unexpected performance decrease.
i was having a conversation with a friend recently about simd in zig (which i have recently picked up and been having a pretty good time with). i find that simd writes decently well, though there's a few weird things:
- some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector).
- a lot of `std.math` is scalar-only (some functions support vectors, though, and i've got a pr open for one of them and plan to do more).
- i'm certainly missing some intrinsics that i get from xmmintrin.h (rcp, rsqrt, few others).
in general though i'm finding it pretty capable.
mitchell, i know you hang around some of these comments sometimes – i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?
I don’t know zig syntax, but wouldn’t it be possible to put this common pattern into a macro and simplify it to mostly a lambda on V?
I was hand-rolling NEON SIMD 15 years ago, and in many cases the compiler (clang/llvm) simply out optimized me. I kept the attempts that were better than what the compiler could already do. That was ARM NEON, quite new at the time, not SSE, and again that was 15 years ago that the compiler could already beat me much of the time. I hate the naive cult coder adage concerning premature optimization, but this might be a situation where you peruse your compiler output before you start writing code in a manner the compiler can for you. In Clang, auto-vectorization is enabled by default at optimization levels -O2 and -O3.
[flagged]
SIMD is great and it's important that people responsible for high performance code understand it and recognize when and how to use it.
It's also easier to get wrong than right if you just dive in because it has a lot of implied constraints and tradeoffs that aren't obvious if you're not familiar with how computers actually work. And frankly, most contemporary software engineers know almost nothing about that (and obviously do fine work anyway).
If "everyone should know" anything, let's start with computer architecture before we jump into vector instructions! That's already a monumental ask these days.