alt Hacker NewsToday I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”.
Stunned to see that Gemini threw its digital arms in the air and gave up.
Replies
It's extremely hit or miss. I've had it one-shot a pretty decent analytic prototype from a brief description, but also had it get trapped in hour-long back and forth regression hell over incredibly simple things like adding a static favicon (ie it would add it, then keep taking it away with every subsequent iteration, breaking something else every time it was asked to put the favicon back etc.).
I just tried this and it worked without issue.
Some considerations:
1) tell it to extra t the data (in a new session) does that work?
2) if it doesn't, could there be something up with the PDF?
As many commentors suggested, this works well with Gemini so there is likely a missing variable in play.
Share your prompt and the PDF and let's see if we can determine what.
That's better than the loop grok got stuck in trying to use git and push the work it did leading to a $15 api credit deduction.
I built a little research dashboard that monitors new papers from specific research labs. There is a paper ingest skill that writes Python scripts using pdfplumber to dismantle pdf’s. I have also used it to fetch supplementary information to replicate/augment the published tables. It can also use plotdigitizer to extract raw data from plots.
The PDF reader for Gemini is extraordinarily poor in my experience. I like the writing style of this model a little better, but for most tasks people would use AI for, Gemini is probably not what you want to be using.
You should just have it OCR a screenshot of the PDF that would probably work better
It does this pretty often. Gemini is an "intelligent" model, but it's massively nerfed and so isn't useful for real work. If you use it with an agent harness, you need to design the harness to detect this and start a new session. Once it nerfs itself it won't try again.
You didn't say whether you were using the App but the App's performance seems to be severely throttled compared to API.
My go-to for this is to screenshot and use the built-in text extraction in the screenshot tool (I'm on a mac), then pass on that text data to whatever processing. It's a pretty good tool so long as the PDF is in OK shape (I've had errors in scanned images).
I haven't heard any accounts of it doing that since Gemini 2.5, but it was pretty easy to get it to do it with a programming task back then after a few failed attempts. Very interesting to hear it'll still do it.
We've been quite impressed with GCP Document AI. Not sure if it has a free tier but perhaps that's where Google is putting all the good OCR.
This comment appears to
1. be made up.
2. have successfully nerd sniped HN
To what purpose I'm not sure.Testing an LLM bot?
Years ago, I used Acrobat to extract tables from a PDF. Had to do it manually, but it pasted nicely into Excel.
I envy you that it admitted that rather than simply making up data and lying about it.
[dead]
That's interesting because my experience has been almost the opposite. A few months ago I tested Gemini on converting screenshots of tables from PDF files into CSV. I tried it on several different tables and it got every one right. It consistently outperformed ChatGPT.