Again, I'd base this on capability. For example, you state: "Memorizing facts is not the same as learning skills." Maybe it's not. I'm not convinced it's not. In the ChatGPT 4 days (or so), I taught the instance I used some chess tactics/strategy that it didn't seem to know. And it appeared to use them correctly after that. Now you might note that this isn't really learning -- they just memorized it and then applied it appropriately.
With a sufficiently large context window, can you make a lot of things look like learning. And if it's not sufficient then point out the capabilities that aren't possible with this approach. To me that much more convincing.
That is ephemeral context. Not permanent learning.
You might ask “why not make the context larger ?” Well, that just spreads out the “attention” focus wider and wider, and you get the “lost in the middle” effect as details get blurred over and not surfaced to results.
You might ask “why not make the context longer-lasting ?” Well, that means the signal-to-noise ratio drops as more and more information accrues over time that isn’t germane to the problem at hand.
As I said, LLMs as we know them today are effectively lookup devices, very very clever ones, but they lack comprehension, and without that there is no value judgement on what is useful (and conversely what is not.) They lack the ability to forget. They lack the ability to create new permanent storage of information. The gap is not some small details, it’s the difference between single-cell and multi-cellular life.