alt Hacker NewsAnd this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.
And this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.
Obviously this article is advocating for spec as source code and not checking in code to keep it around between “compilations”.
Why would you care about reproducibility at all if you were checking in the code as well. If you check in the code, you’d never need to rerun the same prompt.