alt Hacker NewsI think there is a misconception here: CUDA is not even close to being C for assembly used by Nvidia's GPUs, PTX is, and it uses JIT to compile to Nvidia's GPU assembly, SASS. It honestly easier just to have LLMs write PTX directly than to go through CUDA C/C++ at all.
Again, "CUDA" isn't a programming language, it stands for Compute Unified Device Architecture; "C/C++ for CUDA" are the high-level languages that compiles to PTX and then SASS as well CPU orchestration code via NVCC.
And to be honest, pretty much everything you can do in CUDA C/C++, you can also do in HLSL/GLSL compiled to SPIR-V, as long as the Vulkan hardware extension is available.
My point with the c comparison is that it often feels like a very thin layer that still requires you to know a lot about what's going on under the hood. It was more to give a view in what it's like to work with the lib/api.
Maybe I wasn't being precise enough with my language for this forum, and also my last hands on experience with it was roughly 6 years ago, maybe it's gotten better. But it was much less (and forgive the imprecision!) python/pytorch-like where you say hey take this big blob of data and just slice and dice it on your many cores, and more like ok, here is the data, let's cudamemcopy it in these size chunks over to the gpu itself, to be used by this block of threads and run these commands (kernel in cudaspeak) on it. Much more painstaking and micromanagey of the resources.
Pytorch IMHO feels like a proper abstracted API that hides the details and lets you just unleash the fury at the cost of some efficiency, while the cuda api itself, similar to working with C, forces you to really think about the low level details. I have a heavy backend and systems development background, and while it wasn't really intimidating to me, it was like wow you really have to have a deep working knowledge of how these things work and it felt like a step back in time IMHO.
I doubt that's going to satisfy you but I think it gives a clearer picture of what using cuda is like if you typically use higher level languages and haven't touched C since college.