alt Hacker NewsOf course. Models don't actually require VRAM. Nor do they require regular RAM. You could have 1 GB of RAM and swap the model to disk as you need different parts of it. And if you didn't have enough disks you could access weights via a network connection.
Replies
kQq9oHeAz6wLLS • today at 3:13 AM
The key to running in lower amounts of VRAM is patience. It'll be slow, but it'll work.
You don’t even need electricity. You could print the model weights onto millions of sheets of paper, and hire a team of carrier pigeons to fly them into your office one by one. No VRAM!