logoalt Hacker News

johnnyApplePRNGtoday at 2:55 PM0 repliesview on HN

I was wasting hours yesterday trying to get DeepSeek V4 Flash (with Qwen 3.8 27b as the vision agent, actually) to read sheet music to pass a Terminal Bench 3 benchmark and none of it was working... nothing... I changed models to gemma 31b, I tried OCR models... nothing could get it...

And then I realized, wait a second... you're testing the harness not only against a difficult benchmarking problem, but it's one you're literally never going to use the coding harness for either, lol. I don't write programs that read or interact with sheet music and I never will.

tl;dr Being frustrated that a "state of the art" vision model doesn't have perfect vision is a fools errand.

It can read and extract information from screenshots and PDFs just fine (my setup). No need to worry about edge cases.