alt Hacker NewsIt looks like they had a hard time with the magic wand, peacock-to-cat example. Despite using a very long prompt that repeatedly stresses that the video should look photorealistic and NOT like a video game, the output video looks like a video game. The model is clearly capable of generating photorealistic humans and animals, so I wonder why it struggled to do so with that particular prompt.
Replies
It's probably trained in a video game-like environment.
I have no idea if that's the case, but it would be funny if it didn't support negative prompting and they got someone to write the prompt who wasn't aware of that.
Edit: Okay, I read the prompts and they use negative phrasing a lot, so I guess it does work, but boy, are these prompts laborious. Requirements are repeated ad nauseam. This model seems to be quite tone deaf.
Probably too much of the training data that has Magic wands were from a video game.
When people talk about "photorealism" it's to describe something not real (like video game graphics and artwork) that looks almost like a real photo/video. So you don't say that a real photo is photorealistic, and if you use that word when prompting models you end up making it not realistic.
I might not understand what "world model" means, but from my understanding world model is the model trained on real world data, so theoretically having all laws of physics baked in the weights. So perfect world model should not be able to make the video for given prompt, as it is impossible in real world and the only possible way it can appear is in a setting of video game or movie. So this is expected, doesn't mean that is the reason behind this particular generation.