alt Hacker NewsWhat if they’re not pelicanmaxxing, but svgmaxxxing in general?
Because otherwise using a LLM to generate complex svgs is pretty niche and what I thought made this a good benchmark when it was new - generalized programming and spatial knowledge.
Obviously image gen in svg format is not a particularly hard problem if tackled directly on its own.
Replies
I think svg is a balanced test because of the level of indirection and the required 'conceptualization' of physical elements then expressed through code.
Not an expert on this, so won’t speculate regarding what traits would create robustness specifically attributed to SVG visual representation capability, but felt you might find this paper on reasoning models trained on physical world video data becoming better at general reasoning interesting:
If they're optimizing for SVG generation, then that's an excellent outcome in my opinion. Vector images shouldn't be "pretty niche".
Then it's great. A year ago I couldn't get any AI to draw a simple company logo in SVG or even convert from raster. No doubt the Pelicans put pressure on the labs to fix the awful SVG situation. Now I can even make a decent Peli in 3D.
Gemini have absolutely been SVGmaxxing. They've openly talked about it.
Funnily enough, not that niche, because I have tried many times to do it as part of a wider project.
SVG are really useful you can create images that don't have the AI look
I agree, other formats, both textual and binary should be tested.
But that's a genuine worthwhile capability. It's like benchmarkmaxxing on a weightlifting competition by getting really strong.