alt Hacker NewsGPT-5.6
Comments
Funny to see that they did not include Fable 5 in their GeneBench and LifeSciBench comparisons because "it does not answer advanced biology questions and refuses the majority of questions in this eval".
Winner by default!
GPT-5.6 Sol sets a new SOTA on ARC-AGI-3: 7.8%
Sol is the first verified frontier model to ever beat an ARC-AGI-3 game
I've been testing Sol/Terra/Luna now since yesterday, running complex evals on all of them and I feel a bit... mixed on how they perform.
The eval is an agent that runs a set of tools and a prompt we can tune separately for different models. The OpenAI version of the prompt was specifically tuned based on their guide[0]. Then we let Opus to run another agent that acts as a user, trying to solve a problem (anonymized and taken from production). The problem is complex and we don't expect it to be solved by these agents, but we measure how the agents operate when faced with a vague problem:
- Opus 4.8 and GLM 5.2 both identified a constraint sooner and stopped so the user can fix an issue first that the agent cannot solve.
- Sol tried hard to solve the issue with different tools, burning tokens, until finally reached to the same conclusion with Opus and GLM. It was two times more expensive compared to Opus and six times more expensive to GLM for this task.
- Terra went even further and started calling tools that would not solve the issue, burning tokens and failing.
- Luna repeated the same failing tool call until it hit the round limit, and burned more money than Opus.
I'm kind of puzzled with the new GPT. Like, yes Sol is OK for programming, but I was expecting to get a cheap agentic model for non-programming tasks, one that can detect if things go awry and correct. Terra is too expensive and Luna not really fit for the task. Sonnet 5 is a bit better but more expensive than Opus 4.8, which is still the best in my evals. GLM 5.2 is extremely good if you can define the task and the tools clearly for it, and costs pennies!
[0] https://developers.openai.com/api/docs/guides/latest-model
Ok long time Claude Code user here; lately I've started to realize there's other great models out there I should be trying, but I'm hesitant to leave Claude Code behind for something new.
What's the consensus today on codex vs claude code, does it really matter anymore?
I love testing the new models by asking them to code a toy RTS game. Here's what Terra did: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-terra.html (one try, in codex app, xhigh effort)
Comparing this to other models, I find it similar to GPT-5.5 and a bit behind Sonnet 5. You can see how other models fared here: https://senko.net/vibecode-bench/ (you can also fetch the prompt and the the 5.6 Terra resulting code on from that page).
I don't have access to Sol yet (on a Plus sub, which should get it according to what I've read), so can't do the more interesting test. I'll update the above page as soon as I get access - hopefully soon.
I just watched the video on their launch page and I am really not sure how I feel about it. On one side, it's cool that these people get to start businesses and stuff using ChatGPT (assuming these are true stories), but how much of the business is really them? And how much does this business rely on a chat bot always being present as a kind of know it all employee? Maybe I'm just being naive or old fashioned (haven't really used AI much), but seeing these two people who started a cereal business for example talking to their laptop as if they're talking to a human advisor makes me feel, I don't know, I find it creepy.
By the way, this isn't about their 5.6 version in particular I guess, it's just the first time I've looked at one of their videos.
I really wish there was just an easy guide on when to use Sol vs Terra vs Luna, and it just moves further into confusing territory when it comes to naming.
The naming convention is especially difficult to decipher depending on what your native language is. Of course a latin language speaker might be able to easily determine oh yeah each one is slightly bigger than the other but I still think it borderlines too confusing.
That aside all the numbers look amazing, and I'll be happy to probably main this alongside grok-4.5 for a while comparing the two on price and efficiency.
I vastly prefer the direction that OpenAI seems to be going with token efficiency and performance compared to Anthropic who seems to be moving towards a world where you just token-max as much as possible ignoring any and all costs.
I really appreciate the focus on intelligence WITH token efficiency. I'd like to see that become the trend. Smartest per token metrics. Least tokens to accomplish the task above a certain success level. Most of my tasks would benefit from efficiency / token, but switching models constantly, and trying to guess the right model and effort level takes up too much of my processing.
The frontier graph on all these benchmark are extremely in favor of 5.6 Sol over Fable, more than the best model comparisons in previous iterations.
I'd like to know how cherry-picked this is, and what tests it performed less overwhelmingly in, but I suppose that info is not going to be on this post.
If it pans out to be as good as it says, that's great. On the other hand, if this model is not overwhelmingly impressive over Fable, I will lose what remaining trust I had in these announcements.
We Openly hate OpenAI because they’re not very Open but we secretly hope they win against not-open-at-all Anthropic.
"We've extended usage of Claude Fable" message incoming any day now.
The most impressive part is the token efficiency/cost per task of 5.6 Sol, it makes Opus 4.8 and Fable look extremely bad ($1.04 vs $1.80 vs $2.75)[0].
And 5.6 Luna ($0.21) is also impressive, cheaper than GLM 5.2 ($0.37) with higher intelligence.
Just my two cents. I'm on the Plus plan, I ask gpt-5.6 sol / high to analyze a vibe-coded codebase (~50k LoC) and write a plan to make it production ready. It wasn't a great prompt, I just wanted to test it quickly. It ran for ~15min and consumed 95% of my 5h quota (I thought it was gonna crash). The output is excellent but just a heads up that it consumes a lot of quota!
5.6 Terra (mid tier model) as good as Fable on DeepSWE while cheaper than Opus API pricing. Seems like a homerun.
First impression of 5.6 Sol in Codex is fantastic — the model asks dozens of clarifying questions before starting to implement where other models (including 5.5 and Terra) just yolo it with assumptions that needed to be walked back later.
It's working great for me.
I generally prefer OpenAI. I usually start projects with Codex; I love the plan we created. Then, after about 2 hours of working, back and forth, etc.I realize it's drifting HARD, or getting stuck on relatively simple things.
Once I get frustrated enough, I sometimes start over with Anthropic. Anthropic has been better (for me at least) to work with from start to finish.
I have the $200/month plan for each of them. (And I used to have the $250/month plan for Gemini... lol?)
Both Fable and Sol are good and definite improvements. I don't have a favorite yet. It's too early to tell.
The biggest difference I'm noticing is usage.
Anthropic is like a used car salesman. Not allowed to do a basic check on your own car (website), they'll scrape every penny possible out of you. Low limits, high api prices, trying to keep everything in their system.
OpenAI is like a cool but aloof dad. Go ahead and borrow his car, shoot, he'll even pay for your gas every once in a while. He'll answer your 5th question drunk, forgetting what you asked. But, at least he's a nice drinker that buys the group shots.
I have to watch my Fable usage, and I'm sure as hell not going to pay API prices. I don't have to watch/worry about my Sol usage even in Ultra.
But, 1 thing I'm noticing using Sol Ultra (on fast mode) is that it's slowing WAY down after a bit. I work the opposite of peak times so that's not it.
Anyone else noticed the "Extended: Fable 5 is included in your weekly limit through July 12 blablabla" disappeared from claude code? Did they panic-delete the july 12th deadline ?
GPT-5.6 is a really good model, and quite cheap. I can finally replace GPT-5.3-Codex for my Tool Calling in n8n.
Here's my benchmark results for GPT-5.6:
https://aibenchy.com/?q=gpt-5.6
(the high reasoning variants are still running, uploading them soon too)
EDIT: The high variants are there too, enjoy the hamsters[0].
"GPT‑5.6 delivers a step change in design judgment. With only high-level direction, GPT‑5.6 creates tasteful, ergonomic, and functional interfaces. Its stronger computer-use capabilities let it inspect and refine the rendered result—not just generate the underlying code or content—so it can catch visual and functional issues and apply finishing touches before handing the work back."
This one is really promising, as it may allow to close major gap with Claude in design/UI skills
From my first tests today, it is a workhouse. It can scan my whole code base, optimize every part, with a greater level of autonomy than other tools. This is insane, we are living at the best time.
>> approximately 700,000 A100e GPU hours of black-box automated red teaming
Amusing that they use A100e as the reference point to sound impressive. Different ways you could make that conversion, but based on FP4 FLOPs (yes it's disadvantageous to A100, that's the point), that's something like 200hr on a GB300 NVL72 rack.
Not nothing either, but far less astounding sounding than 700k hrs.
Here are 18 pelicans - six each for Luna, Terra and Sol at the six different reasoning effort levels (plus the price to generate each one): https://static.simonwillison.net/static/2026/gpt-5.6-pelican...
Or if you want to see some in 3D, OpenAI featured a pelican riding a tricycle, bicycle, pony and another pelican in their livestream this morning: https://www.youtube.com/live/Wq45rvPGNHs?t=1070s
Things I have been struggling with Fable over and GPT 5.5, were just solved handily by SOL in a real "thank you, next problem" kind of way. Overall, something that just works is way less wasteful for your usage than struggling back and forth for hours.
I find it interesting that no one here has mentioned the increased (usable) context window 258k -> 353k. That's huge, but I wonder if it means we pay long context (2x) for the ones past 272k still.
I have Fable send the specs/plans it comes up with to GPT for review and in 2/5 cases yesterday it found additional 1-2 bugs while in the process of reviewing.
GPT-5.6 didn't try to fix the bugs (as instructed) but it did surface them, which is something that didn't happen with GPT-5.5. When spec/plans approved Fable sends back to GPT-5.6 for ralph implementation and it seems to be an even faster, more reliable workhorse than it already was in GPT-5.5. Overall, impressed. Will continue to be a core piece of my workflow.
I am looking to rent an apartment in a new residential tower. I have asked Fable and Sol to scrap the listings from various sources, deduplicate them and present them as a web application. Just using the cowork/(ex-)codex application interfaces.
Fable had issues with the sourcing and organizing images, and shoot itself at foot looking for shortcuts as usual. As I was getting it fix these back and forth, I copied my prompt and gave it to Sol.
Sol has surpassed my expectations by far. With a one shot simple prompt on a complex task, it gave me a working web app with everything I want with minor issues to track and fix.
Unfortunately, I'm finding that in long-form agentic use, when I'm trying to use Sol, I keep tripping guardrails – moreso than even Fable, somehow.
I don't know exactly what part of my codebase is triggering it, so I'm going to have to keep poking, but apparently the guardrails are not that gentle despite the phrasing. :(
Wow the video is much better.. the PR spend clearly went up a lot. Mainly just showing "real people" doing "real stuff".
I haven't tried an OpenAI model for a long time, but with Fable going to API pricing soon this might be enough to get me to try codex.
Still fails my internal test of counting legs on animals who have had extra legs photoshopped in. However if prompted to determine what is wrong with the image, it does get it right.
This kind of "out of bounds" image analysis seems to be a very difficult problem to solve, but totally necessary for transformers to really bring about massive change.
5.6 Sol High Fast is using more capacity than 5.5 High Fast, I hit the 5h limit for the first time.
Other than that, I think the difference between 5.5 and 5.6 will be the same as 5.4 and 5.5. 5.5 is just less frustrating to use, although not perfect and still has derp moments. But a lot less than 5.4.
So I expect 5.6 Sol to be smoother to use. But so far it just feels slower. We'll see.
I flip back and forth between whoever currently has the more powerful frontier model that isn't cost prohibitive - subscriptions only, API pricing a non-starter. Today that's Fable 5 which has been excellent, as soon as it's Sol I'll switch to that. The OAI/Anthropic harness behavior has mostly stabilized for me with consistent AGENTS.md that I sync with CLAUDE.md - I like pi (pi.dev) and have tried to build it up to get performance comparable to the two "first-party" harnesses, I'm just not there yet.
One major sticking criteria for not going with OpenCode / pi for all of my coding is I want access to the tier-1 frontier model of the day without API pricing - e.g. afaik I can't use Fable 5 via pi harness even though I have a subscription, so for this week I'm on Claude Code. It's not the need to Fable 5 for everything, but even if I just want the marginal intelligence benefit to stress test an architecture decision, it's a safety blanket to know there isn't a ~smarter~ model I could have used. And for my use cases, the doggedness and capability of these frontier models has been insanely effective.
My feeling is we're still in the Uber era subsidy period - the moment the subscriptions either try to lock me in longer than a month or stop OAI/Anthropic stop delivering frontier models in the subscriptions, I'm out - switching fully over to pi.dev or another OS harness and routing my token spend via OpenRouter or offloading to Qwen locally. Then I'll have to put an accurate dollar amount on frontier intelligence.
Huh, a good alternative just as anthropic's 50% weekly subscription subsidy is ending this weekend. Time to see if it's benchmaxxed or actually a strong leap over GPT5.5.
They also seem to really not care about alignment, or care about it in the wrong way. It's entirely missing in the blogpost and there are some concerning bits in the model card, seemingly treating CoT controllability as something to be "investigated" rather than the warning sign it's supposed to be.
In the introduction video they say 5.6 Sol autonomously post-trained 5.6 Luna. Curious what this means.
Benchmarks look really promising. Suspiciously good, even. I guess we’ll see soon enough.
My question to previewers: how are the guardrails for random joe that wasn’t personally blessed by the ai pope to access the non-nerfed model? Fable is a nightmare in this regard, but I’m not sure whether 5.6 also gets a critical side-eye from the gubmint when you ask it to fix bugs in your code (you filthy hacker, you).
GPT-5.6 is the first model where I’ve actually frustrated to use it.
I’m explicitly telling it to do something extremely specific and it’s just not listening to me.
Eg, I gave it an image to update. The image is sized 400x200 pixels. It then generates a new image at 300x300. I explicitly state to be 400x200 in size and it won’t listen.
> average daily output tokens per active researcher were more than twice the highest level observed for GPT‑5.5.
Very interesting: I wonder if the RL approach is diverging between Anthropic and OAI?
I noticed that Fable uses shell tools almost exclusively (even to search and edit files), compared to previous Anthropic models.
Having run some experiments with 5.6, I notice that it uses built-in file systems and provider native tools much more (not shell tools), compared to previous OAI models.
Not specific to OpenAI / Codex, but I'm curious what people are doing to protect themselves from any destructive actions by their coding agents? Just install and pray? Explicity approve all actions? Reconfigure for safety? Run in a sandbox (Docker) ?
I’m interested in knowing how each of GPT 5.6’s variants fare in non-English writing/translation tasks.
GPT 5.5 has a tendency to write English calques and non-idiomatic prose in other languages. Although that can be somewhat tamed with detailed instructions and a corpus of confusing terms, the model’s output often reads like a literal translation rather than native prose. Since I notice these issues most clearly in languages I know well, it makes me reluctant to trust the model’s output in languages in which I’m less proficient.
Ironically, ChatGPT began as a simple text-generation tool, but much of its offerings and benchmarks now focus on coding and agentic workflows, while leaving behind what made it notable in the first place.
Is any of those comparisons about Pro vs non-Pro (Pro is only available in $100+ plans)? I am curious about that but I think Sol, Terra, Luna are different sizes of it without the Pro part, and I want to know how much worse do I have it on the $20 plan compared to if I upgrade.
They talk a lot about speed in the article, but having tried out Sol today with Pi, 'medium' mode, one thing that stands out is that it's really ssslllloooowww.
It also defaults to 'low' mode for some reason. Can't tell if that's a step backwards compared to GPT-5.5 in medium mode so I'm sticking to medium.
Edit: just noticed it's spawning subagents in 'high' thinking mode.
I use 5.5 a ton. It's immediately apparent that 5.6 is truly a better model. Hope they don't lobotomize it later.
I can't try it since it hasn't appeared in my Codex yet, but this is is necessary from OpenAI in my opinion. Fable is just so much better at understanding broad context. I only use GPT 5.5 for straight forward easy to describe tasks, and it does crush those. But I spend a lot more time steering Codex towards good design on broad concept type tasks, ones that Fable shows sometimes surprising clarity.
I look forward to seeing how it compares once I have access. Not getting tripped by spurious safe guard flags could be an advantage.
Just used terra ultra for exactly one prompt in codex and it ate through my full 5h window in about 10mns (20$ plan). The results look pretty good though. Luckily I have had my chatGPT subscription for a while and have a bunch of resets available (nice compared to anthropic).
Assuming I take the 5x plan it would give me about an hour of active sessions with terra ultra (maybe ultra is not good value regarding tokens?), not even using Sol yet. Does everyone using codex use the 200$ plan?
I normally use the 100$ anthropic plan and barely ever reach the usage limit.
8% on ARC-AGI-3, they actually got some traction going...
I would be really interested in real life throughput. For an agentic chat situation, we are still on 5.4 - not because of the cost, but it's simply much faster than 5.5 with comparable results. Also we are using gpt-5.4-mini a lot for quick summaries, tldrs etc. In an ideal world we would upgrade 5.4 to 5.6 terra and 5.4 mini to 5.4 luna. But does somebody already have some measurements at least in terms of speed?
I've found Sol's propensity for delegating to subagents can make it... disastrously expensive, especially with each subagent having some implicit floor on further reasoning/context gathering before action.
The base model is certainly cheaper and more token efficient etc, but on large tasks cost in some way is now n^2
The developer's guide (https://developers.openai.com/api/docs/guides/latest-model) has some interesting semantic tips for using the model:
> Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
> Original image detail: GPT-5.6 preserves the original dimensions of images sent with original or auto detail instead of resizing them to a patch budget or pixel-dimension limit.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
> Control warmth: GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic.