Thanks!

I think this is a great question. I have some thoughts on this but no hard evidence (neither does anyone else!)

Your argument relies on the AGI system being the model arch + weights. I think that the weights are irrelevant. The training algorithm is what is AGI: You choose/find a training set that covers a task, and then some form of deep learning with a big neural net.

For LLMs (which is a weak kind of AGI), this is freezing a model after NTP pretraining + RL postraining. This allows us to do well on a wide distribution of tasks.

But if we had a scoped task, I think its possible to just do the same thing with (1) a smaller dataset that covers this task + (2) not freezing the model (ie. test time training). This is easy for ARC because the data is small and has been curated well yes, but i don't see why can't this apply to more complex/ill-defined tasks (obv lots of things unsolved to make it work today)

> even if the ARC train+eval sets contain the sum of human intuition between them, superficial differences IRL would render your model unable to identify which examples are relevant to which problems, and thus unable to transductively reason.

In this hypothetical world, the dataset becomes incredibly large, and training on it makes it close to an LLM. You can then finetune transductively and we get the same thing (other approaches already do this iirc)

--

Note: BTW I'm not claiming that this is an AGI system, the benchmark creator also was clear that ARC is not sufficient for AGI (his goal was just to point out unsolved stuff, and the stuff ARC-2 pointed out was demonstrated very clearly by large reasoning models).

In the above work, I just wanted to show that AR transformers without pretraining can perform really well on this benchmark, which was incredibly non-obvious before.

The claims in this argument are separate and I haven't proved them yet