logoalt Hacker News

kamranjonlast Saturday at 7:23 PM0 repliesview on HN

They're basically second place behind NVIDIA for model inference performance and often the only game in town for the average person if you're trying to run larger models that wont fit in the 16 or 24gb of memory available in top-shelf NVIDIA offerings.

I wouldn't say Apple isn't serious about AI, they had the forethought to build the shared memory architecture with the insane memory bandwidth needed for these types of tasks, while at the same time designing neural cores specifically for small on-device models needed for future apps.

I'd say Apple is currently ahead of NVIDIA in just sheer memory available - which for doing training and inference on large models, it's kinda crucial, at least right now. NVIDIA seems to be purposefully limiting the memory available in their consumers cards which is pretty short sighted I think.