logoalt Hacker News

inkcapmushroom • yesterday at 9:24 PM • 10 replies • view on HN

For several of these examples the details change if something goes out of frame and returns. A second castle is added in the dragon one. The people walking away from the camera change in the train platform one. The Hong Kong moped rider doesn't have a helmet and then he has one.

It's very good at maintaining consistency for things that stay in frame, but I don't know that this points to it developing a strong "world simulation" like they claim.


Replies

cvoss • yesterday at 9:50 PM

It's like its simulation model is confined to the renderable area. At the cost of increased computation, it would not be hard to generalize the technique to rendering a sphere centered around the POV to keep some of those artifacts at bay; however, the fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object. For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.

> While it can maintain coherent environments over short sessions, visual details and object permanence can still shift or drift over time—especially as a simulation runs longer, after multiple complex prompt edits, or when navigating throughout the scene. Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are active areas of ongoing research.

➕ show 5 replies
rao-v • today at 12:24 AM

I don’t understand why these world models aren’t designed to be backed by an updatable world state. It could probably just lay out the world onetime and reference it as needed vs trying to invent from wholecloth via diffusion. Object persistence!

monk_grilla • yesterday at 11:18 PM

> It's very good at maintaining consistency for things that stay in frame

I noticed several inconsistencies in things that stayed in frame. See the wall on the bridge in the Van Gogh example as they walk past it to see the other side.

daniel_iversen • today at 12:34 AM

I bet it's configurable and just a matter of how much resource you want to allocate. If it's truly a world model then it seems to me that it's precise benefit is that it can keep track and order in things and they just choose to accept some noise/chaos/randomness/? for the demos when things move out frame? Does that sound sensible? I'm just guessing but don't feel it's a hard limit.

human305893 • yesterday at 11:16 PM

And these are the examples they chose. I have to wonder if they add some with mistakes on purpose.

stavros • yesterday at 10:09 PM

It just doesn't have object permanence yet!

Alex13192 • today at 12:44 AM

[flagged]

gubidonius • today at 12:18 AM

[dead]