I wish I knew what to pay attention to. I've always had trouble with that. I spent 2024 and 2025 learning how neural networks and transformers work. The conclusions of that learning are pretty sobering. Everything uses transformers and despite all the novel architectures that have come out in those years, transformers are still the best and I'm not sure how to come to terms with that.

Does it mean that researchers wasted their time on useless dead end architectures, or are they ahead of the curve and commercial companies are slow to adopt them?

Even the coding agents are more primitive than expected.

>>when the first Go grandmaster was defeated by a "pretend intelligence."

A computer playing GO is intelligent now? Is this the kind of conversation we're having?

>>I sure wish I had.

And how would you have changed your decisions in those last 10 years if you did?

>>The next best time to start paying attention is now.

I am paying attention, I use these tools every day - the whole idea that they are intelligent and if only you gave them a robot body they would be just normal members of society is absurd. Despite the initial appearance of genius they are just dumb beyond belief, it's like talking to a savant 5 year old, except a 5 year old can actually retain information for more than a brief conversation.