The funny thing about LLM is, you can build one from scratch and yet you still won't understand how it works. You would understand what kind of matrix multiplications the neural network performs (in fact that's not that hard. An OS is orders of magnitudes more complex), but you would still have no idea why it does what it does.

I'm not sure it's possible to have intuition about systems that work in thousands of orthogonal dimensions. In fact I'm pretty sure most of the research is people trying fairly arbitrary things and testing them and then post rationalising implied understanding of what is really happening on top of good outcomes.

I think you're conflating complex and complicated. An OS is very complicated but is predictable. An LLM is complex and not necessarily understandable.

That depth requires math and traditional computing depth. Where is the time to be young and have that depth?

> A lot of people here are responding to the message but not to the meaning.

Well it is framed as quite specific advice.

(I'm done with mining PG tweets for meaning)

What corpus would one be advised to use for training? Should one limit oneself to legally downloadable documents or should one follow the examples of Altman, Amodei, and Zuckerberg and use LibGen and Anna's Archive?

Agreed. During comp sci we got to re-implement various algos of networks, OS, database, firmware.. and it all gave complimentary intuitions that were useful when tackling practical implementations and bottlenecks.

> understanding the bare-metal firmware for a computer

IMO this is still relevant, everything surrounding the LLMs needs such a vast infrastructure that I don't know if I would find it more useful to learn the maths behind ML than CS

The next class of problems to solve for future generations is how to survive on a hot planet and how to recover from the damage done by reckless lunatic profiteering assholes.

Couldn't have said it better

Depends, the assumption things are predictable always negatively affects both Market Bears and Bulls alike.

Indeed, if credulous folks look to the world expecting people to bestow success upon them... than the disillusionment with reality will hit their savings harder.

The Shrek movie market correction correlations are undeniably funny, and a new film is due July 2027. OpenAI may be going public in the next few months while still losing $2.25 for every $1 of customer revenue, and with 6 other firms sharing over $4Tn in debt disclosed to investors in a footnote.

There is only one direction things can go at the Peak of inflated expectations. Popcorn ready. =3

https://en.wikipedia.org/wiki/Gartner_hype_cycle

> "Build an OS" wasn't a common university project because we were all expected to go […] helps you deeply understand how to intuit building for a whole class of problems.

Sure, but it was for a specific degree with a syllabus that taught you the foundational knowledge. It was not expected from the law students to learn how to build one.

> It would be a good idea for young people to deeply know how these programs work.

It would be a good idea for _everyone in the industry_ to deeply know how LLM training, inference and "agents" work, not least because it removes the ability of shysters to bamboozle with bullshit.

But, as much as a good idea it is for the young to understand this, it's the elderly who will be really taken advantage of if they do not keep up - just look at Facebook for good examples of why.

[dead]