alt Hacker NewsYeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds. Would be unfortunate if 4G dropped out under some trees while using the API after all.
Replies
Marha01 • today at 5:25 PM
> Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds.
When the models stop improving, we will get model-specific ASICs that are much more power-efficient.
Power usage isn't an issue. 10 kW is 13 HP. The size, price, and fragility of the components is the issue.