alt Hacker NewsI test models' ability to code by asking them to vibe-code a small game[0]: a Warcraft clone, a Minecraft clone, and a biplane dogfight: https://senko.net/vibecode-bench/
Fable 5.2 doesn't look materially different (if anything, the graphics look worse), but it added more depth to all three games. For example in the warcraft clone, since the goal of the game is to explore 100% of the map and there are lakes, it decided to add an airship unit so the player actually can explore the entire map. Or, since there were guard towers, it decided there should be hostile NPCs otherwise the towers don't make sense.
And in Minecraft case, it decided not to follow Minecraft style too closely (IP infringment guard, perhaps?) so it added a round sun (!!!), two moons (also round :) and a nicer night sky. The rest of the graphics are decidetly worse tho...
[0] not a scientific test, I know; but it gives me an intuition about how a new model behaves, roughly, and it's fun!
Replies
That's a fun benchmark. Amazing way to experience the progress of models over the years.
I started playing one of the RTS and built an army, only to realise that I was alone on the map...
I've had the same 'not looking materially different' experience with a 3D application that is not a game, that I asked Fable 5.2 to improve.
It looks like no breakthrough over Opus 5.5 and I'm somewhat disappointed.
During my use this evening, 25% of the Fable limit on the $200 plan was consumed. I'd estimate it at 3x faster consumption than Opus 5.5.
Nice benchmark. I feel Opus 5.5 is still way ahead of Fable 5.2 at coding outcomes. Your FlightSim game is a good example.
wow, that's pretty impressive
This is really cool, good job dude!
[dead]
The OpenAI results made me realize that the OpenAI models really do love adding flavor text everywhere. The way every corner of the Voxel splash screen was covered in stuff like "Unlimited possibilities, go explore!" cracked me up given how much every time I need a very very basic site made it'll somehow end up with so much of that kind of writing covering every corner of the page unprompted.