The math example on Qwen was based on 50 batched iterations, and the fix was essentially to remove a direction in activation space that causes looping instead of an exit, this was already known to be a thing.

So it's not able to insert new knowledge, but it's able to adjust the routing/looping/weighing of existing one.

As I understand, it can replace post-training like LoRA/GRPO, but if it'd cheaper the authors don't go into

At some point we have to accept that there is enough training data on "being good at thinking" and that being good at thinking generalises really well.

There doesn't need to be specific narrow training data on every conceivable task.

One consideration is that Qwen3.5 has bigger/later versions that are already better at math. If it was indeed training on ML research, it's possible that what it is actually doing is applying lessons learned from the Qwen3.8 27B weights (which independently solved the problem) to Qwen3.5 0.8B.

I doubt it's that simple, but I could totally see an LLM being able to make that association and apply a fix from an improved LLM to a smaller one with the same architecture.

You don't think an LLM can perform tasks not in its training data?

>It just doesn’t line up with my understanding of how these models work. Is there really enough training data on direct weight editing to give a model this skill?

I mean, this question indicates that your understanding of how these models work is, indeed, lacking.

Despite what some people having being hollering about, treating LLMs as just "stochastic parrots" isn't doing their capabilities any justice. They are constantly exhibiting emergent behavior which wouldn't be found directly in their training data. They are capable of combining different ideas that they are explicitly taught to perform novel tasks.

I'm not saying you're guilty of this, but the people claiming LLMs just regurgitate existing data don't understand how LLMs, and likely their own cognitive experience, actually works.