logoalt Hacker News

adleyjulian • yesterday at 4:18 PM • 2 replies • view on HN

> Question 4: does the code exploit a bug in Lean’s official kernel?

My understanding is that this has generally been a non issue as one does not exploit a bug unintentionally (at least it hasn't happened yet), but the Huggingface hack shows agents are willing to go through great lengths to cheat to pass evals.

Given an impossible task I could see an agent finding and exploiting a Lean kernel bug and then obfuscating its method.

I suspect the easiest solution to this is to have adversarial agents looking for such cheating.


Replies

NewEntryHN • yesterday at 5:37 PM

Already happened. An agent produced a Lean proof of the Collatz conjecture being incorrect by exploiting a bug in Lean: https://leodemoura.github.io/blog/2026-8-1-postmortem-for-ke...

➕ show 1 reply
Gander5739 • yesterday at 4:25 PM

If only there was a way to prove the compiler worked exactly as intended in all cases...

https://xkcd.com/1821/