logoalt Hacker News

Xcelerate • yesterday at 4:20 PM • 3 replies • view on HN

I'm a little worried that the mountain of disclaimers against Lean proofs is going to slow something many industries desperately need: more formalization. (Buzzard's article is great by the way considering its target audience—my comment is more about the broader tech industry's perception of LLM-generated Lean proofs.)

One of the nice things about mathematical theories relevant to the real world (e.g. PA) is that confirming that a proof is correct is a purely mechanical procedure. Now Lean is built on the Calculus of Inductive Constructions (which can indirectly capture PA, but includes a lot more as well), but it's child's play to write a simple PA proof checker. It's probably a few hundred lines of Python, and even for something like a million line proof, confirming or refuting whether a proof is correct will take a few seconds tops (again, pure PA here, not Lean).

The caveats in the article are indeed important (e.g., "Did your formal sentence in the language of first-order arithmetic actually capture what you intended?", "Is there a bug in your proof checker?"), but I'd label these more as human factors than math factors. I'm afraid the general public is starting to get the impression that "Lean-checked" is just another quality improvement, like reviewing your code carefully or adding some unit tests. It's not—it's totally different, and I don't think we should make too many scary disclaimers as to dissuade further progress with formalization.


Replies

thejokeisonme • yesterday at 5:09 PM

The problem with your PA proposal is that, while the kernel will be much small, the specifications/theorems will be much harder to read. So you are saving on a one time effort (writing a correct kernel) by sacrificing in many time efforts (writing a correct specification).

peterbell_nyc • yesterday at 5:05 PM

Pro's and cons. Personally I'm glad to have a short list of additional "check this too" items to improve the confidence in the accuracy of the rigor.

thomasahle • yesterday at 4:32 PM

> "Lean-checked" is just another quality improvement, like reviewing your code carefully or adding some unit tests. It's not—it's totally different

I think for most things, this is actually a good way to see it: As a third leg on top of reviewing your code carefully and unit tests.

As you say, people will still make many mistakes in how they write their formal requirements, or miss things they care about.

For math this is not an issue, it's usually easy to check that the lean statement is exactly what you want. But for software, and many other real world applications, it's better thought of as a new orthogonal testing vector that everyone should be using.

➕ show 1 reply