> “LLMs are Bad At Games”

> Any time I tried to use an LLM for a game-y thing, it would fall over.

You know it's funny. I did a bit of LLM-assisted game dev this year.

I spent a few weeks working on a simple Pong game, just polishing it, making small changes here and there. Frontier models couldn't change or add a feature without breaking something else. It was still more productive than working "solo" but quite annoying.

Meanwhile, I asked a model to make an entire game -- about 100x more complicated -- and it just one-shotted it perfectly, no issues.

I'm not sure how both of those things can be true at the same time. Something something "in-distribution" ?