They do learn in context, and very sample efficiently. Continual learning is active area of research and we sort of already have something resembling it with persistent context. So yes they do learn.
I consider that to be the illusion of learning. You are not wrong, I think they may actually learn in the future though. But not today.
I consider that to be the illusion of learning. You are not wrong, I think they may actually learn in the future though. But not today.