alt Hacker News> Which makes me wonder: how do these models calculate probabilities?
By unprincipled methods. To calibrate confidence in an answer you need a task specific calibration set. You can't pre-calibrate for tasks unknown ahead of time, which is the case for most uses of decision models.
In some sense everything an LLM does is by unprincipled methods. But it works! We judge it empirically post hoc by its performance rather than relying on tuning.
Hell, human thinking is unprincipled in basically the exact same way.