I am curious how this things perform compared to a "classical" neural networks. Tasks like image classification, feature extraction has been studied extensively and there were a lot of very cool applications before (medicine included). The biggest obstacle was always data but even that could have been partly automated with some semi-supervised learning schemes. Of course the Bitter Lesson applies here, given a big enough model, it can outperform specialised ones, but at this point it's not clear 1) By how much? 2) If this is even true in the first place or if we have the means to train much leaner networks

It is becoming a boring homogenous world where we are trading deep expertise with being able to query the magic box

There is no "final barrier", the goalpost will continue to change until you are not aware of what is going on anymore. This is already becoming true as the recently produced math research is not fully understood, CoT moves further away from looking like human reasoning and datasets become generated rather than collected from existing human data.

Now I'm curious if anyone has or might try to create an LLM that lacks all knowledge of the post-einstein/newton/plato world and see if it can figure things out. I imagine lack of training data would be a problem, but maybe it could be a way to reverse-engineer an effective prompt that can give us insight in the platos of our world?

Too bad the pyramid was already mapped, just got lost in Mexican archives and wasn't accounted for. There's already literature on it.

Couldn't it be plain old AI, not an agent?

[dead]