logoalt Hacker News

simonw • today at 6:16 PM • 11 replies • view on HN

Pelicans for 6.2 Sol: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

And for 6.2 Astra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

Here's an updated comparison grid for the entire GPT-6 family: https://static.simonwillison.net/static/2026/openai-pelicans...

UPDATE: And if you're bored of pelicans, here's what I got for:

> Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data

6.2 Astra (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

6.2 Sol (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

For comparison, here's Fable 5.2 Medium: https://tools.simonwillison.net/yonkers-rises and Gemini 4 Carbon: https://static.simonwillison.net/static/2026/yonkers-mech-st...


Replies

vopi • today at 6:23 PM

This might be in other pelicans, but 6.2 Sol/Astra is the first time I'm noticing a model shade the leg further in the frame darker to mimic depth (or a shadow).

It's also interesting you can clearly tell 6.2 Astra does a much better job of the above. 6.2 Sol attempts it at higher reasoning levels, but it's crude in comparison.

I wonder if it comes from the (seemingly) recent inclusion of 3D environments in training data.

RGS1811 • today at 6:35 PM

The emergence of the beach setting and the scarf is interesting.

I_Write_It • today at 6:24 PM

They all seem very good, with a lot of attention in detail ! Thx for the Pelicans!

globular-toast • today at 6:47 PM

Do we know why they always end with a scarf?

➕ show 2 replies
adverbly • today at 6:26 PM

What's with everyone saying these Pelicans are great?

These are some of the worst regressions that I've seen in a while.

Pay close attention to the stacking layers around the feet, pedals (sometimes there aren't even two pedals), wheel, and two legs. Some pretty obvious mistakes in most if not all of these...

Edit: And what's up with the single arm on the two highest Astra images?

edit2: really? I'm getting downvoted because people don't like that I pointed out a mistake in a Pelican svg? Is that where we're at?

➕ show 2 replies
firemelt • today at 6:21 PM

any index link for all ur pelicans i want to see fable and opus

tintor • today at 6:21 PM

Sol 6.2 max is still failing on pelican benchmark. Left leg is jammed in chain.

Astra 6.2 max did a similar mistake on the left bicycle pedal.

➕ show 1 reply
BoldColdHold • today at 6:23 PM

Wow the yellow is leaking into their llms too.

Hacktrick • today at 6:19 PM

I'm not sure if this is a useful benchmark any longer

➕ show 1 reply
p11p • today at 6:19 PM

Are you getting any signal out of this anymore? All I can see is "good pelican" and "dogshit pelican". These ones look pretty good.

➕ show 2 replies
EugeneOZ • today at 6:20 PM

still the best benchmark! thanks