alt Hacker NewsI think you could do many performant things without any involvement of software. For example you could do AVX on RAM. The CPU would recognize PID RAM and offload AVX instructions to the module.
Then, by simply asking for a special memory address you could have access to registers/regions within PID RAM that serve as a result region.
Let's say you would need to run a sum over megabytes of data like for accounting. You could just ask RAM to do it and load just the result. The bandwidth would could be 8x higher and software could stay the same.
Doing scalar operations, frequent dereferencing and similar would not get much peformance benefit in many cases, as loading and accessing CPU cache is often much faster. But simple vector operations over large data could be massive.
Having accelerators on RAM like for jpeg compression, audio decoding or mass data operations could be beneficial but you would need to be careful with heat dissipation.
Personally I'm a big fan of the "in-ram accelerator" idea especially for server space. Doing fuzzy search in RAM could be massive performance improvement.
Replies
Data movement and local operations are still bottlenecked today on memory bandwidth. Butterfly primitives, sorting/fft/1D-convolution, the whole cub library, could be ported there and have great performance wins. But the pain of programming and maintaining code using this...
> run a sum over megabytes of data like for accounting
It's been many decades since the last time somebody ran a sum over megabytes of data for accounting and though "damn that's a bottleneck I need to optimize".
> Personally I'm a big fan of the "in-ram accelerator" idea especially for server space.
The operations this model supports are so extremely limited that you would be hard pressed to find applications where it's worth it.
> Having accelerators on RAM like for jpeg compression, audio decoding or mass data operations could be beneficial but you would need to be careful with heat dissipation.
I think we are essentially reinventing SSE, AVX & friends from first principles. This is already being done. Compare the speed of libjpegturbo to a non-vectorized implementation and you'll find a 2-4x difference in throughput.
If Infiniband does this for MPI on the network and realizes Sun's "The network is the computer" dream, I believe we can do this for other parts of the hardware, as well. Not only for AI, HPC will love this idea.
Yeah this sounds a lot like a natural evolution of SIMD for me, just cut out the middleman and put the SIMD units straight into RAM.
I can imagine some power savings for always-on display applications too. Rather than periodically waking the CPU/GPU to update the frame buffer, you can just stash small bits of periodic logic in memory (e.g. move the second hand of a clock).