alt Hacker NewsIs coding "solved?" I've been using Fable and Grok to vibecode a library to improve text extraction for legal PDFs. (Which have things like line numbers in transcripts that messes up text extraction.) This is a simple, dumb problem that should be like killing a mouse with a nuke. But I got to "pretty good" with a few iterations of not looking at the code or design. The program works, pretty well. But the design and heuristics are only 80% of the way there. Now, the models are just "churning" on the existing design, making ad hoc changes struggling to get to the last 20%. It seems like I'm going to have to turn my brain back on and think about what I actually want to do, which I was trying to avoid.
It's better than writing assembly! But it still seems to be in the realm of "a new kind of tool" even for the actual coding part, without even caring about what the code looks like or security or anything like that.
Replies
What harness? And why not just Fable+Opus?
Personally I use only one family for code editing, other families for code navigation.
Models tend to reason about the code using their own intuition and putting other families onto the same codebases may end up with them getting confused. Esp if that other model is as unhinged as Grok.
Also, harness matters a ton. Use a wrong harness with a good model and you’ll get terrible results.
Without knowing details of your approach, I would venture that your challenge is not with the coding per se but extracting structured data from PDFs. It’s a surprisingly hard problem because PDFs are optimized for preserving the visual structure and layout of the content for precise rendering and printing… NOT for preserving the logical structure of the data!
Which is why the best results these days for extracting structured data from PDFs is by having the model do it directly rather than writing code to do it. It literally takes that level of intelligence to be reliable at it.
A common approach is to provide the model with a template or structured schema describing the format you want the data in, and the PDF itself, and it should return a JSON with the appropriate values filled in. It won’t be 100% but probably higher than what you’re seeing now.