alt Hacker NewsWas the temperature 0? Cause unless I don't understand it right, any non-zero temperature implies probabilistic next token prediction.
You did mention, seed, which I haven't seen available anywhere else (in my limited experience) and it can explain a reproducible inference result, but I feel like this matches the "letter of the law" meaning of deterministic rather than the "spirit of the law".
Replies
bnjemian • yesterday at 1:46 PM
Using a PRNG and a seed, you can have paeudo-randomness and determinism. The pseudo-randomness is the key piece, it’s random to the extent that it cannot be distinguished from true randomness in polynomial time classically.
bruce343434 • yesterday at 2:09 PM
Perhaps they just used the exact same random seed everywhere?
If you send the seed along, you can have a non-zero temperature: when it's time to select tokens, a PRNG will be seeded and used for the selection.