alt Hacker NewsAren't these specs already outdated? Seems meh. I would think players would like to bring something new to the table instead of bringing what was okay two years ago or so.
Give me 1TB RAM and 5TB/s or something that exciting.
Replies
simoncion • today at 6:42 AM
> I would think players would like to bring something new to the table instead of bringing what was okay two years ago or so.
Speaking of... from [0]:
Introducing Qualcomm Cloud AI 100 Ultra
High performance and cost-optimized cloud AI inferencing for generative AI
Written by
Rashid Attar
Nov 13, 2023
Today we announce the Qualcomm Cloud AI 100 Ultra, a new member of our portfolio of cloud artificial intelligence (AI) Inference cards, purpose-built for generative AI and large language models (LLMs). ...
tempest_ • yesterday at 8:43 PM
5 year old GPUs are still at crazy prices.
Plenty of businesses want in local inference and are priced out.
➕ show 1 reply
throwawayffffas • yesterday at 11:41 PM
This is all the CPU companies playing catch up with Nvidia and AMD.
The thing you are describing is much closer to AMD and Nvidia latest generation server offerings.
See MI455X for the absolute cutting edge and MI350P for something you can use outside of a datacenter.
For most purposes, this seems reasonable? This looks like a single-height card, so fitting four of these in a workstation or server would give 512 GB RAM and 2 TB/s total bandwidth. That's pretty attractive for a lot of current LLMs, even if one wishes for 1 TB capacity at 5 TB/s bandwidth.