alt Hacker NewsI posted this in the other Astra thread but it's just fallen off the homepage, so...
Pelicans from Astra, plus 5.6 Sol, Terra, Luna for comparison: https://static.simonwillison.net/static/2026/gpt-6-and-5.6-p...
I think this is a genuinely interesting comparison grid. Astra may be more expensive, but if you have a budget of 10 cents for a Pelican Astra low gives you something SO much better than the other models.
Astra uses less tokens overall too, for better results.
Astra transcript here: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Replies
It seems to start understanding how bicycles work. Look at the front fork. There are reasons why it's shaped the way it is in real bicycles. If you do physical simulations in 3D with reinforcement learning you will start understanding that most of the arrangements that the other agents use just won't work.
You would not expect the developers of the model to optimize for a well known benchmark?
Simon do you have a page somewhere that shows _all_ of the penguins created by various models across time?
You’ve been doing this public service for so long (well, for so long in “AI hype” years anyway) that it’d be fascinating to see the evolution of this artifact across time.
I find curios that Astra's pelicans are basically the same (yellow sun top right corner, green bike, same bike shape, same legs style, very similar background) while in other there is more randomness.
Astra pelican looks amazing: it looks like it was made by a professional artist. All the others look like they were made by kindergartners.
Interesting that all of the bikes are turquoise colored, except for the medium effort
Oh my goodness. I was not prepared for Luna on "none".
Reminded me of https://clocks.brianmoore.com/
Crazy how much the Astra pelicans resemble GLM 5.3's (https://crimson-jeri-74.tiiny.site/), down to the color of the bike and the scarf.
The improved front fork design mentioned by Threatripper is about the only thing Astra is doing better, IMO.
Do you have other ideas of "combinations" for this kind of benchmark?
I'm wondering if this is being trained on by the models today.
radial spokes on the back wheel is not a thing irl. i wonder if any model has gotten that right?
You’re closer to this than I am but do you find it odd that helmets are almost never included? Biking is almost always accompanied by helmets
Fallen off the homepage so what? I genuinely don't understand
[dead]
[dead]
[flagged]
Btw, they had 3D pelican-on-the-bicycle easter egg in one of the promo videos: https://youtu.be/bOC3DisEOfg?t=117 so I'm pretty sure that they spent some small amount of resources to train the model to produce good svg version as well. :D