Let me stand here on the Skeptic's Corner and be skeptical, so that the users who complain about skeptical comments have someone to direct their ire at. You're welcome.

Right, so, first, I haven't looked at the proof. Graph theory is not my subject and it would probably take me a few days to get my head around the whole thing. If OpenAI's LLM was used to prove an important graph theory result, then that's very good for them and graph theory.

However, I have to note that it's been 52 days since 20 May, the last date that OpenAI announced their previous mathematical result (a disproof of the unit distance conjecture).

What have OpenAI been doing all this time? I am willing to bet a good percentage of my money that they were trying, and failing, to produce the current result, or possibly something even juicier (one of the Millenium prize problems maybe?). They are hell bent on showing that their models are good for maths and science so they're very unlikely to have sat there twiddling their thumbs until they suddenly sprang into action and prompted their LLM once to generate just one proof. They must have been running the thing constantly, multiple instances of it, over that entire period.

Going by the instruction to run for eight hours before returning or giving up in their released prompt [1], that means they could have made at most 156 attempts to solve this problem, each of which failed except the last one [2].

So what happened to those other 156 attempts? Are we ever going to see them?

More importantly, who was it that selected the announced result? Who decided that this result is an actual proof? Until now, every proof generated by an LLM has been verified either by human mathematicians, or by human mathematicians x a proof assistant. What happened this time?

Obviously, any claims that this result were produced "autonomously" must be evaluated according to the answer to that last question. So far, LLMs have been incapable of distinguishing between a correct and an incorrect proof, which is also why they need to be run multiple times until they generate a correct one. If something has changed, it'd be interesting to know.

Finally, a magic eight ball that's correct one time out of 156 may be useful; or it may not. I honestly have no idea. I think time will tell.

__________________

[1] "Spend at least 8 hours on this before even thinking of returning or giving up"

https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98...

[2] That's 52 days from 20 May, times 3 for each eight-hour attempt in a 24-hour day.

But note well that the X post says that the solution was produced in "just under one hour" so that means the model didn't really stick to the prompt's time limit. Which means there may have been considerably more than 156 attempts that we'll probably never know of.

Or even considerably more if the model ignored the time limit going the other way.