alt Hacker NewsWith a 4B parameter model that probably ran through 8GBs of RAM multiple times to run.
At a certain point we should seriously talk about CUDA accelerating Postgres instead.
Replies
bt1a • yesterday at 11:28 PM
pardon but aren't disks usually the bottleneck? im all for CUDA acceleration and CUDA accelerating culture
➕ show 4 replies
eloisius • today at 2:33 AM
What would you accelerate? Is there a lot of linear algebra you could throw cuda at in Postgres?
➕ show 1 reply
I would think it's possible to make it so that the 4B model only needs to be called during an initial phase, and then the same queries it constructed can just be re-used with values replaced, unless you're generating a lot of unique on-the-fly query shapes.