alt Hacker News> GPUs aren't really SIMD, they're SIMT (single instruction multiple thread)
False. If they were threads they'd have their own PC. They do not - only the warp has a PC.
> You just write plain scalar code and the hardware model dispatches it efficiently to SIMD execution units.
Absolutely not. If you don't write coalesced loads, bank-conflict free, predication-free, cooperative code you will get worse than CPU performance.
Replies
Actually not so false anymore. (But still they don't expect you to use this knowledge while coding, and you should treat all threads in a warp as moving in lockstep)
> In GPUs of compute capability 7.0 and later, independent thread scheduling allows full concurrency between threads, regardless of warp. With independent thread scheduling, the GPU maintains execution state per thread, including a program counter and call stack... [1]
1: https://docs.nvidia.com/cuda/cuda-programming-guide/03-advan...
>> GPUs aren't really SIMD, they're SIMT (single instruction multiple thread)
> False. If they were threads they'd have their own PC. They do not - only the warp has a PC.
They are using the term SIMT as it is normally used[1]. The "single instruction" part means that there is only one PC shared across multiple 'threads'.
[1] https://en.wikipedia.org/wiki/Single_instruction,_multiple_t...
Yes, of course writing naive code assuming each lane in a thread group is a real thread is going to cause problems, but I didn't feel like I needed to go into that level of detail replying to someone just learning about GPU internals. I tried to cover this loosely by mentioning how you need to know how it works for maximum performance.
If you want to get more pedantic you also need to look at your target hardware and their specific micro-architectural quirks and features to get the best performance. AMD specifically benefits a lot from exploiting the scalar unit over the vector unit, you save loads of register file space if you can keep data in SGPRs over VGPRs. There's lots of traps you can fall into where you can load data from buffers into SGPRs but they get promoted to VGPRs because the scalar unit lacks an opcode for like one math operation you did to the value somewhere.
While each lane isn't truly a thread because it doesn't have its own PC the programming model definitely tries to make it seem that way. The threads can terminate at different points too. And again, the ISA isn't a vector ISA. Your register values are scalar.