alt Hacker NewsSo if you dig down a bit it turns out the author had been trying to solve that problem for a year with GPT 5.4 and 5.5 and he fed all that information to the prompt he gave to Sol Pro which may or may not had direct access to the author's chat history. So the claimed "148 minutes" was really "a year plus 148 minutes".
Moreover, it seems the prompt included the technique used to solve the problem:
https://old.reddit.com/r/math/comments/1uxj3cy/after_openais...
In the prompt I basically just throw all reasonable approaches at it, without making a big distinction for what to explore most, and these approaches would all be reasonable for someone who knows the area. Sol helped me with the prompt as well, for which I gave it the CDC prompt, some ideas and specifications, a crystal clear problem description, and then modified things slightly myself after. One thing I do wonder is how much it accessed memory of previous chats, since as mentioned I had worked with 5.5 and 5.4 on this previously, and the main construction is not so different from something I discussed there. But, the function class max of affine functions that worked in the end was also in my prompt, so I'm not totally sure.
So it's not clear to me the degree to which "GPT-5.6 used a prompt" to close the gap etc, or the author basically did all the work himself and assigned it to GPT-5.6 out of enthusiasm.
Replies
> Moreover, it seems the prompt included the technique used to solve the problem:
I don't believe this is true. The author sent techniques he used, but I don't believe any of those were ultimately what GPT-5.6 used.
GPT-5.6 also provided the Lean formalization, which was not provided at all by the author.
sounds like a feature, not a bug
it might imply using weaker models to attempt the problem first is a good supplementary prompt to a more advanced model trying to do the same
I (author of the original post and paper) can add a few things here: 1. My previous approaches with GPT 5.5 were really not very sophisticated in terms of my input. I threw the problem at it, and just kept encouraging it to go iterate through ideas without any success. 2. The approaches that are in the prompt, though they will seem cryptic to someone not in the field, are relatively natural ideas. In fact, the construction that worked was something that even 5.5 initially looked at but was just too weak to see how to make it work. From my view, I would have never gotten this result myself. Imagine you are telling a contractor to build the empire state building, and you say: "You should explore approaches that can include building materials like steel, wood, concrete, or clay, and any combinations of those. You can use arcs, columns, supportive beams, and anything else you can think of to solve load-bearing issues. Do not stop until you've completed a viable plan to construct the empire state building." And then the contractor shows you the finished empire state building using reinforced concrete and steel beams with all kinds of crazy ways of making everything stable; that's kinda how I feel.