alt Hacker News> But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs.
Not unique as in: no one makes their compilers deterministic, and you have to work to make a non-deterministic one. LLMs are non-deterministic by default, and you have to contort them to the point of uselessness to make them deterministic
> If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc.
Again. Even if you control everything, the only time they produce deterministic output is when they are completely neutered:
- workaround for GPUs with num_thread 1
- temperature set to 0
- top_k to 0
- top_p to 0
- context window to 0 (or always do a single run from a new session)
> no one makes their compilers deterministic
Go (gc) was designed for reproducible builds by default, so clearly that's not true, but you are right that it isn't the norm.
Even the most widely recognized and used compilers, like gcc, clang, even rustc, are non-deterministic by default. Only if you work hard and control all the variables (e.g. -frandom-seed) can you make these compilers deterministic.
It's fascinating that anyone on HN thinks that compilers converge on always being deterministic or always being non-deterministic. I thought we were supposed to know things about computers around here?