> We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive.

The “Attention Is All You Need” paper came out in 2017 and OpenAI released GPT-1 in 2018. ChatGPT was released in late 2022 but that was not the beginning of LLMs. Transformers and generative AI go back even further.

> It seems quite premature to say that. We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive.

When we were 3 to 4 years into mobile phone revolution it was already abundantly clear that it's changing the way people leave.

Ditto when mobile phones became smartphones, and for many other tech items we got in the past decades.

As for LLMs... outside of a few precious professions you could live your life and not notice they are there.

It's clear that while LLM progress is continuing, it's at something like a linear rate, or slower.

The singularity requires exponential improvement. That's not happening. Maybe we will develop something that sets that off, but there's no indication of that right now.

As a vocal advocate for the use of machine intelligence in certain contexts and a skeptic of claims they are useless or always hallucinate or whatever...

It hasn't been even close to even the recent claims! It's been like three straight years where it was supposed to utterly transform society any minute now.

I think we more or less have a fragile consensus that in like, computer programming and really very little else, that on a good day you're probably going to come out ahead with the AI assist. That seems relatively uncontroversial now. But it also seems like success with AI is largely about working hard to get good at it, as a first order concern. It is not at all obvious that blind, uncritical use by anyone is a net win.

It's pretty unclear, sone might say dubious, that anyone has made serious, aboveboard money net of debt and equity, other than the hardware vendors. There's some pretty serious revenue, but it's a drop in the proverbial bucket against the outlay. There like a two trillion dollar balance sheet hole in the US alone where investment into AI has gone.

While I personally don't agree with them, multiple S-tier machine learning researchers think we've got our wheels stuck in the mud, that autoregressive decoder architectures on a tokin-suffix pre-train is tapped out as a paradigm.

It's ok to say "AI is starting to get useful in pretty durable ways" and not sound like an Anthropic shareholder/employee, i.e. completely full of shit.

[dead]