alt Hacker NewsI read somewhere that the expected 3-5 year lifespan of inference GPUs was not holding up to scaring: in practice, the GPUs have the same lifespan as any datacenter CPU or RAM. I think that estimate was based on Bitcoin mining ASICs, which wasn't an apples-to-apples comparison.
Replies
thesuitonym • today at 2:47 PM
I'm not certain it's the longevity of the actual hardware so much as there is a huge incentive to replace your hardware with faster, more capable equipment. Who wants to invest in the AI company that's using 10 year old GPUs when the next company is using brand new?
➕ show 1 reply
Older GPUs still have an active secondary market but they are probably not profitable at market rates for inference because newer chips are so much more efficient.
If Semi-Analyses is correct [1], even H100s would lose money at current prices - I doubt that is actually true of H100s in practice because from what I can tell it is based on a 3-year depreciation from the new price, but the point is that there is a point at which old hardware's operating inefficiencies dominate.
https://inferencex.semianalysis.com/infx/profit-estimator/de...