Yesterday I was trying to add a patio to some real estate photos, i.e. "virtual staging".

I uploaded 4 different photos of the same house, from different angles.

I asked Gemini, ChatGPT, and Grok add the same patio to each photo. The results were laughably incoherent (not to mention ugly).

I repeatedly asked them to make the 4 photos consistent in 3D, and it's obvious that they have no way to do this -- no understanding of the scene.

They also could not understand the concept of "back patio" and "side patio" -- relative to the house, when the house was being photographed from different angles. Even when I pointed out which one was the side patio, they would get it confused in a different image.

(I should not be surprised by this, and yeah I know everyone says LLMs have "jagged" capabilities. But to me this shows how bad all the extrapolation of AI I read on say Hacker News has been)

I think they do but only when it's applied to learn some specific spatial task.

We can see for e.g. that frontier models have got really good at CAD sorts of tasks recently. That suggests benchmaxxing RL to a degree but it does mean they've suddenly got quite good at forming simple geometries in a step by step way.