alt Hacker NewsI was curious to see how open weight models would do on this task so I passed in a screenshot of your source of truth and here's what 2 of the best code-generation models that allow image inputs do:
Inkling (not too great): https://cdn-uploads.huggingface.co/production/uploads/608b8b...
Kimi 2.7 (really well, esp. note that this is the predecessor model, not the latest Kimi3): https://cdn-uploads.huggingface.co/production/uploads/608b8b...
Here's how I tested them: https://huggingface.co/spaces/abidlabs/vlm-screenshot-to-web...
https://huggingface.co/spaces/abidlabs/vlm-screenshot-to-web...
Not bad at all, and this is pretty consistent of what I've found from the current open source models. I haven't tried it with kimi 3 yet, that's on my todo.