I have not looked carefully but it seems like this is over-promising on avoiding catastrophic forgetting.

The "trunk learning rate" is set at 0.1x the learning rate for the experts, so learning on different subjects disproportionately happens in the experts, and the trunk portion is comparatively more stable. But the population of experts can grow and shrink:

> The pool grows when it is short of capacity and shrinks when parts of it stop being asked for.

So:

- doesn't the trunk then _eventually_ still undergo catastrophic forgetting, it just may take much longer?

- and before that point, catastrophic forgetting happens in stepwise chunks whenever the expert pool shrinks?