alt Hacker News$100 AI Music Video: Claude Fable 5 vs. GPT-5.6 Sol
Comments
Unsure if it's just the way they prompted it / coded it, but the output is far too much a literal direct copy of the lyrics. The best music videos have a story arc on the theme of but often not litearlly the lyrics, and start with obscurity and reveal something (following all the literary/story mechanisms)
Consider Amber Run - Found lyrics versus the video, and the story arc of the video
AI is destroying the economics which allowed for a sizable middle class of artists. The issue is that many are paid for their art mostly for its aesthetic rather than artistic value. This isn’t the most creatively fulfilling, but it previously allowed many artists to make a living while refining their skills, often enabling them to pursue their real creative ambitions on the side.
These are awful. It’s like Suno music. Seems convincing if you half listen. As soon as you pay attention you notice all the cracks.
Interesting how bad this is when you don't use video models and your own direction.
The first two clips are made with Kling (not affiliated, but I use it myself): https://xcancel.com/PJaccetturo/status/2076312902685085815#m
Obviously not one-shot, and finalised in a video editor. It's quite doable to get this kind of fidelity.
One thing all models aced was the costume design. I don’t know why this was the case. Maybe they trained exclusively on lots of fashion data to use models in fashion industry.
Filmmaking is inherently an iterative process. You don’t normally one shot music videos in the real world. Agents should review and iterate at every step of the process. Lots of room for improvement here.
Also for comparison, budget of the original Uptown Funk music video was probably around $100,000. Even “indie” is pretty expensive in traditional filmmaking due to high cost of equipment, people and places. 100x cost reduction is huge for any industry.
> None of the music videos were great
Glad they acknowledge this.
Curious how much time in addition to tokens this costs. If you have to spend $25 and wait 45 minutes to get a basically unwatchable video, I'm not worried about indie film makers being replaced just yet...
When the line was "don't believe me just watch"
And then the clip was literally just an arm wearing a watch!
That's freaking hilarious!
It's like someone playing charades
Tangent: philosophically to me, art is inherently human. What makes art meaningful and impactful isn’t whether it looks good or cool. It’s the story of the artist, the context of the art itself, the hard work and struggle involved, the meaning represented by a human creating something very specific to their own personal context and taste. Or a mix of any of that.
Can AI be used as a tool to help create art? Absolutely. But as a rule, I do not give any shits about AI generated content like this. It’s not art. It’s not human. And the line is really how much meaning and effort _a human_ is putting into it.
If a human spends a minute or two prompting AI and then tweaking The result, and peddling it as their own art… get outta here. You made some content. That’s easy, and no one should cares. Content can already be shoveled out faster than we can watch it with or without AI.
Meaningful art is not mass-produced, generated content.
I realize art is completely subjective, so some person may find meaning in AI generated art. That’s fine, and that’s part of what could make that art. (Like an original way of presenting something that really resonates with someone.)
But this garbage ain’t that.
And I realize this is just a capability test, but plenty of places will see this as cheap and good enough. But it ain’t art, and we should push back against another cost-cutting measure that does nothing to make the world better.
It's an interesting experiment and the results are surprising. I will say that if you're a musician I'd bet anything you can make a way cooler and better music video for $25 and 45 minutes with your friends.
I found these hilarious!
I was ROFL over some of the mistakes the AI made. Parts of it was amazing, other parts were amazingly off.
Well if AI destroys bad music videos by making them cost $25, I say it's a win. What will be left will be seriously scripted productions.
Watching the videos and reading the methodology, it feels like "scientist one-shot music video with LLMs" which is... useful but in no way represents how one would use the models.
If nothing else, it serves as a great reminder that you can't offload taste and creative decisions to AI. AI is also not a mind reader. It _may_ do a useful thing sometimes - but it's much better to codify that thing in your requirements if it's something you want to occur consistently.
This is how you do AI music videos: https://www.youtube.com/watch?v=Njk2YAgNMnE
Lean into the not-quite-but-almost uncanny-valley-ness. Make it a feature, not a bug.
It is jarring to me that most of the dancing seems slightly out of sync with the music. It is like a music video uncanny valley - images look good, but the lack of sync to the sound shatters the illusion entirely.
These videos are far from good, but it's possible to create a decent AI music video if you use a human-in-the-loop workflow: https://www.youtube.com/watch?v=52Km88uAZ4M
Interesting article, but those videos aren’t. Not even pre-MTV quality.
While checking out the gallery, came across this image:
https://www.tryai.dev/gallery/d3725e9b-1df9-4c10-8a23-2fc705...
There’s something wrong with it, but I just can’t put my paw on the problem.
I don't have a problem with AI and can't stand the anti-AI brigade, but... this is the worst thing I have ever seen in my life.
This specific type of garbage is exactly what arms the anti-AI critics with valid arguments. We should really wait a few years for the technology to mature before releasing these kinds of projects into the cultural sphere.
Woah that sucks. Closed the tab when I saw the cartoonishly stupid, completely out of place smoking thermometer.
A music video is not (necessarily) a depiction of what the lyrics say at any moment. Soulless for sure.
This absolutely is the opposite of what I want to see in a music video
The result depends mostly on two things: The first is the LLMs ability to come up with a good story, choose and prompt the video model, evaluate and editing. The other is the quality of the video models themselves.
The LLMs mostly used older video models from a year ago that are capped at 10 second single shot clips. Newer video models like Seedance 2 go up to 15 seconds (Seedance 2.5 even up to 30 seconds) and they support multimodal references.
As a tool, I think those video models can be pretty amazing for artists as you can create things not possible before. Here is a music video made by an artist using Seedance 2: https://www.youtube.com/watch?v=yYfDJQxyHFg
All these AI discussions seem to lack the nuance that the real utility of these tools will come in the finer points.
GPT? Sure, you can write a book with it, but in writing its use as a tool by a real human is perhaps best when it is providing a jumpstart to creativity. If I'm writing a script I might have a chatbot give me a rough outline which will usually be complete garbage that I throw away 90% of but it got me over the decision paralysis of getting started.
Image generation? Sure, you can create varying degrees of body horror wrong-hand imagery and ugly-as-sin event flyers. But it's also indispensable for, as an example, object deletion and cleanup workflows within Photoshop. Or intelligently letting me change the paint color of a car while preserving the reflection's tones and realistic shadows.
Video generation? Sure, you can make ugly nonsensical slop but I imagine it will be hugely useful for AI driven rotoscoping, quickly creating placeholder or background resources.
Of course, the ethics of how this training data was all gained creates an entirely different aspect of discussion. Supposedly Adobe's image generation, by far some of the worst on the market, is so bad because it is ONLY trained on imagery that they already have reproduction and usage rights to. Which tracks, because it does an admirable job of erasing an ugly building and revealing trees and a field behind it, but anything that results in it having to generate stuff more complex than the landscape photography that makes up the glut of its training data has... hilarious results.
Not a fan really. There are certainly uses for AI, but this is lacking something. Personally I think it works best as a filter or applying a style or effect that is difficult otherwise, or generating fancy ambient or abstract textures etc.
People constantly think the music video I created for Zingara's Unlock Your Keys [1] is AI, but it really is just real footage all around, except for a handful of lines / pulse textures that were created in TouchDesigner.
I am really excited for the possibilities that AI can give us in the future but often I find trying to use it generatively I run into the paradox of choice and end up paralyzed!
1: https://xfeeefeee.net/unlock-your-keys/ tribal dance fusion music video, sfw but does show some skin. Uses lots of slow motion ink in water footage for texture as well.
I am shocked at the good quality of the output. Might not be great from an art standpoint (according to the other comments), but that's just a question of time (not a lot)
The results are pretty bad for a top stars music video. But it's pretty impressive for not very important short clips like insert shorts of cheap Ads.
I'm scared because that's the most labor intensive area. Top talents will survive but all the volume industry will collapse.
I think these videos are a great way to visualize what is wrong with the "personality" of LLMs: they are shallow, unoriginal, overconfindent.
The videos convey this in a more vivid and direct way than any text answer could. I often have the thought that if Claude were a human colleague, I'd avoid having to work with them.
OP really figured out the secret to getting on the frontpage. Three frontpages in a row lol
I did let Claude Code Opus + OpenRouter API Key (limited to 25EUR) create an an Arthouse Video about "Is AI art or can it go?"
https://m.youtube.com/watch?v=E6XofKqFYeQ
I like the scene with the hands.
Good creator of AI music videos (with human editing etc) is "ALFfx Visual DJ"
Example:
https://www.youtube.com/watch?v=4lWArXcsxYo
https://www.youtube.com/watch?v=pn-4lIFhybA
https://www.youtube.com/watch?v=9kjUEan_fgM
Most of their videos have a similar retro-futurism look
Please repeat this with models from MiniMax:
https://www.minimax.io/news/minimax-hailuo-23
(No affiliation, I just want to see how well they work).
I wonder how much worse this would be if instead of starting with a human-created song: https://en.wikipedia.org/wiki/Uptown_Funk
Start from an entirely AI generated song, and have an AI generated video. We apparently already have an example of such:
Some tells, the face, reflections on faces, multiple legs, no sense of gravity
There's definitely still room for innovation in custom use cases with AI. I like to write and draw comics, but it's very time consuming to make a finished product. Working with tools in default and you'll have a bad time. You have to really guide it and if you're using a longer story to adapt, it'll compress things and lose context during Thinking. Luma labs was the most interesting tool I've seen so far, but there's still a lot of room for growth
Probably a good example why you shouldn't outsource creative work to AI.
For what it’s worth I tried an experiment where I had a similar harness (where LLMs competed head to head playing snake) and made them aware of their budget and it had very little impact. This was cheaper open weight models
Always nice to see that it still really struggles with the nuances of human movement and self-interaction. Especially when it comes to clothing overlapping.
AI videos as in remotion based videos look much better imo since it can code much better than it can prompt for videos with a coherent narrative
https://youtu.be/uDAeAuYyl0E (parody of Claude announcements) https://youtu.be/cSsVNtGPOIg (recreating a fireship video)
All videos are so full of stereotypes and clichés, often borderline (or just plainly) racist. But of course, that's not the main worry for people making these things, right...
The GPT ones are strange. The $25 fable one to me is subjectively better than the others. The $100 fable one is too literal and robotic.
The jevons paradox is you need auteurs to curate vignettes or effects and cut or mask them in etc. That's not really different philosophically when software entered art in other ways. I could see errors/glitches lowering in time but I doubt there will be much acceleration.
Seems like if you build some more scaffolding around it, it wouldn't be bad. I think AI video isn't quite there yet so you probably would want to lean into that. For example you could ask for an animated or cartoon music video so the real shots don't look weird. Also if you gave it some guidance on what a good music video is like it would probably help as well. But yeah idk may be that's not the goal here.
Regular music videos (including the writing/recording) can easily go into 6 figures. I wonder what the $200,000 AI music videos looks like.
We still need a different approach to create this. Consistency is the problem in video, a model should create from start to end instead of different shots.
token cost for the $100 video was $3.25 for Sol 5.6 and $25.05 (!) for Fable 5
for the $25 video it was $4.27 (Sol) vs $16.99 (Fable)
this seems like an interesting benchmark of a complex task, even if the output has little artistic merit. the efficiency of Sol is impressive
The cutting edge technology is being tested with pelican drawings and video creation. You may argue that this is just testing, but actual mainstream use of this tech won't be far off from these tests.
Spice 1 released an AI video for Jealous Got Me Strapped. Got a young AI Spice 1 and Tupac. I see on his channel he put out another AI video in 2024.
The fable $25 version was the best.
It's amazing that someone used those awful videos as a showcase for anything. :)
No need to take any pot shots at this trash in terms of aesthetic efficacy in earnest, if y'all know what I mean. But as an experiment, it's interesting. There seems to be an odd consistency to all of them, which reveals a kind of internal logic & coherence of sorts that courses throughout any iteration.
Aesthetically, where can you place these? I feel like the late David Lynch could have used some tropes in his unnerving, unsettling dream sequences (a la the intro dream in "Mulholland Drive" where people are so cheery and upbeat that it feels viscerally disturbing).
That's the consistent feeling that I'm left with watching each one, is just a deep, deep unsettled and uneasy feeling. A scowl on my face the whole time trying to make through each one.
I did crack up seeing a martini glass presented to the camera, then float in place on it's own and the presenter resumes holding it with his other hand.
Everything else made me nauseous.
Like most of this stuff, it's obviously impressive technology compared to what existed a few years ago. But the end product has zero artistic value. It's a grey goo of the average of every concept picked up from the concept of the song.
A talented creative with a vision could make something more interesting and enjoyable in an afternoon with a $0 budget.