logoalt Hacker News

alephnerdtoday at 5:13 PM1 replyview on HN

> The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's

I disagree. But more critically, I'd argue it's the legacy of the PDP project that led to what became foundation models today.


Replies

HarHarVeryFunnytoday at 6:02 PM

The PDP project was very early - relevant in term of neural net history of course, but hard to see much there relevant to today's large models other than Hinton's reinvention of SGD as an alternative to the layer-wise training that was then the norm.

One interesting thing to note from the PDP handbook are mentions by LeCun and Hinton of what would later be called CNNs, which LeCun claims to have invented. It seems that Hinton deserves just as much credit as LeCun, and in any case these are discussed just as locally connected models using shared weights as an optimization.