It's like looking at the early internet and then suggesting teenagers should write browsers as their projects instead of webpages.

That seems like a perfectly good use of time for a kid in the 70’s! You’d learn a lot of engineering skills and demonstrate a tenaciousness that most people don’t have. I don’t think the goal is to predict what will be the important technology in 10 years. The goal is to challenge oneself with hard tasks and learn interesting stuff. A lot of the “AI” people today were compilers people yesterday

There are more career opportunities building rockets than designing rockets. Lots of welders, machinists, electrical engineers etc build rockets, and those skills transfer.

The same is not so different for AI. A few people design novel AI, but there are a lot of people training AI (especially if you include fine tuning) and implementing AI, even as a hobby.

Building an LLM covers a lot of CS fundamentals and forces you to do lots of research in order to implement one, especially on more modest hardware.

How far into the future can you see?

But he’s not saying that it’s a career opportunity.

It seems to me like he’s saying that doing this thing would be 1) fun and 2) a great way to become employable in the future. I don’t believe he’s saying that this project would be some kind of job training exercise.

When I was 17 and still witnessing Apollo Moon launches, I wanted to build an AI that would handily outperform LLMs as we know them today.

But that was way back in the early 1970's and all I had to work with was a mainframe.

Well the mainframe itself wasn't bad, the real show-stopper was that I didn't own the computer outright, no strings attached, no debt, etc.

>I'd probably try to make an LLM that I could use on some specific problem.

I thought so too back then, still do so I guess this is one of those things that could stand the test of time. I always wanted to start with something a lot simpler than a Moon mission myself. At 17 I already had a significant breakthrough in the chem labs and it was from alternatives to a single processing step plus everything that descended from that, rather than trying to tackle a much more complex detailed multi-step synthesis. I was only 17 but I was not trying to be a slouch, I don't think pg was either at that age but his advice is not for just anybody. I couldn't have done it if I hadn't made major progress since being 16, and it really emphasized at the time how much maturity can make a difference. My imagination ran wild as I extrapolated :)

In a reply from LeCun to pg:

>>I'll figure out a set of methods and architectures beyond LLMs that can quickly learn to perform physical tasks as efficiently as humans and animals. That last item is also what I would if I were 30, 40, 50, or 66 years old

I see no reason to stop at 66 either ;)

But I figured that people owning more computer power than I could ever afford were going to be doing something like this as soon as they could, without having to wait for something like an LLM to arrive before getting peoples' attention.

It did seem like things were going to take longer than you expect, so it's pretty good to have a lifetime of concentrating on the specialized natural science domain expertise, focused now for 50 full years on how it would combine if AI ever got good enough.

Both the natural science and the AI need to be a major cut above, I still see dramatic room for improvement in my own work. If I'm going to have to rely on "other peoples' AI" then that natural science component is going to have to pull a lot of weight to keep up with the kind of computers that only rich-as-hell high-rollers have access to.