> If we don't see something that's substantially better in the ~30B param space soon - it would appear we might've saturated that size with knowledge.

I wouldn't be quite so pessimistic. We may have saturated the current approach, but I think there's a lot still left in terms of compression, attention, active parameters, caching etc. etc.