> Have you seen it reliably do this?
Lol, no. I don't need to. I can predict such things.
For example, I predicted about 15 years ago that AI would decisively beat humans at math, because math is formalizable. Most people didn't believe it until this summer, and some don't believe it still.
To people who don't understand that AI improves over time, my prediction skill seems magical.
But math is not formalizable (ask Goedel why). And no, computers have not decisively beat humans at maths, that's a laughable assertion. Sure, laymen always imagined math as being about proving theorems, but there's a saying among mathematicians: good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians write definitions. But maybe you already thought they've solved math when they were able to calculate a zillion digits of pi, lol.
It sounds like you're a bit like that AI stuck in the datacenter, confirming their hypothesis without needing to actually check whether it's true or not in the real world. Then proceeding to imagine how the rest of the world is wondering at your imagined magical capacity of prediction.
Seems like a non-sequitur? "because math is formalizable", I agree it makes sense to then predict that data-driven AI models will be able to perform well on it. But where is the evidence that software coherence is formalizable? If it were then I would agree, but to me it seems that coherence in a software system relies on a ton of intuitive reasoning and familiarity with company culture, past failures, etc. it's like the least formalizable thing there is.
AI improves over time, sure, but it has differences and advantages and disadvantages over human intelligence. It's just a fundamentally different thing. In the same way calculators are superhuman, and improving, but can't write books. You yourself acknowledge to some degree the uneven nature of that improvement when you referenced formalizability.
> For example, I predicted about 15 years ago that AI would decisively beat humans at math, because math is formalizable. Most people didn't believe it until this summer, and some don't believe it still.
That is a laughable quote for anyone that know the history of type theory and the foundational crisis that happened at the beginning of the last century. Math as a whole can't be formalizable but we can create various formalist for different math purpose just like we have so much programming languages.
And the study and the creation of those formalisms is not something automatisable.
15 years, though. Meanwhile, work wants me working today.