alt Hacker NewsThe interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions.
This yields for exciting ideas or workarounds that might result a post-crisis memory boom (hopefully) also for local machines.
1. Lowest, Apple is evaluating new Chinese manufacturer which means change of supply demand if indeed it has reasonable QA. (https://www.ft.com/content/f4ac5c92-03be-4499-b16a-017a7e9ee...)
2. Companies tries to workaround performance - suddenly single channel is 'ok' ? :) (https://www.gigabyte.com/press/news/2403)
Replies
I would love it if we started designing software with hardware constraints in mind again.
Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.)
There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient.
These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles.
So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.
Necessity is the mother of invention.
> suddenly single channel is 'ok'
Single channel RAM surely beats any disk-based swap.