logoalt Hacker News

a2ff6eeb0yesterday at 6:13 PM2 repliesview on HN

Sure, but I don't know what GCC's behavior is, and I don't vet behavior differences between compiler upgrades. As long as the output works, why does it matter that the black box is deterministic?


Replies

electric_toucantoday at 12:56 AM

It doesn’t matter how exactly GCC works, as long as the behavior is deterministic and consistent (GCC is likely maintaining backwards compatibility between versions, so the behavior of your code likely hasn’t changed). In that case, you can reason about the behavior you need and write your code appropriately.

In the case of LLMs, the behavior is non-deterministic and inconsistent. If I don’t explain how handle an edge case or give a performance constraint, the LLM will still produce code and may do so in different ways, handling edge cases differently and with different performance characteristics. I can’t reason about how the LLM will fill in those gaps, it’s “random.”

Maybe you don’t care about how the LLM handles those edge cases or handles performance, but that’s different than a deterministic abstraction whose implementation details you don’t care about, but whose logic and performance is deterministic and consistent

rogerrogerryesterday at 6:35 PM

Because _someone_ has vetted the output of GCC. It’s used in flight-critical stuff.

The closest thing we have to vetting LLMs is “whoa look, it escaped this sandbox, that’s prolly not great but it’s so cool!”

show 1 reply