logoalt Hacker News

hgoel • today at 3:50 PM • 0 replies • view on HN

Order of operations affects the result in floating point, even moreso at lower precision. GPUs are massively parallel devices with many different sources of latency, so order of operations is not guaranteed for things like dot products, where many threads contribute to the result. Consequently, you can get non-deterministic outputs even though nothing else has changed.