logoalt Hacker News

markasoftware • today at 7:16 PM • 7 replies • view on HN

Have you tried doing this recently? For my work on software at least, the latest LLMs have a very low false positive rate for bugs. They almost never hallucinate issues. My understanding is the same applies to math.


Replies

root_axis • today at 7:28 PM

I'm not arguing that LLMs aren't useful tools for mathematicians, I'm questioning the quality of discourse that amounts to "I pasted this into the LLM and it said there are errors".

➕ show 3 replies
mahogany • today at 7:30 PM

I would guess that LLMs are much worse at math without having a theorem prover to validate approaches with. Since this draft is not formalized, an LLM is operating purely on the text which is more tenuous.

gregorygoc • today at 7:31 PM

I’m doing it daily and it finds a bunch of issues which are not issues in practice.

shmoil • today at 7:28 PM

Pre-chatgpt 5.5 it hallucinated quiet a bit on hard math. Starting from 5.6 it is incredibly good.

simplyluke • today at 7:51 PM

I suspect LLMs are much better trained and more reliable at bug identification in react apps than draft-state proofs on the bleeding edge of math, on problems that are specifically highlighted as extremely difficult to solve without error in their training.

acedTrex • today at 8:14 PM

> the latest LLMs have a very low false positive rate for bugs.

Man people live in different worlds, astra for me finds 25/6 bugs all the time.

SubmarineClub • today at 7:56 PM

Cool story.

I have the opposite experience in my line of work.

➕ show 1 reply