alt Hacker NewsMy point with the c comparison is that it often feels like a very thin layer that still requires you to know a lot about what's going on under the hood. It was more to give a view in what it's like to work with the lib/api.
Maybe I wasn't being precise enough with my language for this forum, and also my last hands on experience with it was roughly 6 years ago, maybe it's gotten better. But it was much less (and forgive the imprecision!) python/pytorch-like where you say hey take this big blob of data and just slice and dice it on your many cores, and more like ok, here is the data, let's cudamemcopy it in these size chunks over to the gpu itself, to be used by this block of threads and run these commands (kernel in cudaspeak) on it. Much more painstaking and micromanagey of the resources.
Pytorch IMHO feels like a proper abstracted API that hides the details and lets you just unleash the fury at the cost of some efficiency, while the cuda api itself, similar to working with C, forces you to really think about the low level details. I have a heavy backend and systems development background, and while it wasn't really intimidating to me, it was like wow you really have to have a deep working knowledge of how these things work and it felt like a step back in time IMHO.
I doubt that's going to satisfy you but I think it gives a clearer picture of what using cuda is like if you typically use higher level languages and haven't touched C since college.
I think you are pretty precise and your comment is appreciated, so I think I need to clarify my wording as well. I think a lot of CUDA C/C++'s difficulties are self-inflicted because they conflate two things into what they call "kernels": runtime and shaders. Fundamentally, GPU programming for compute is easier than either graphics programming as well as CPU programming because you are limited on what you can do on the GPU, memory allocation is static because the matrix size doesn't really change dynamically during runtime and branching behaviors are generally to be avoided for GPU compute.
It's pretty heretical for me to say this, but a lot of GPU compute complexity that Nvidia is doing in CUDA is unnecessary and is by the simple fact that to do anything meaningful you have to either use their library or handle allocation/scheduling yourself. Imagine if JavaScript required you to handroll part of the V8/Node's JIT compiler, allocator and scheduler yourself every time you just want to make a webpage, that is essentially what CUDA is doing.
The actual "program" that runs on the GPU, the compute shaders in PTX/SPIR-V, are very low level but pretty straight forward once you get down to it.