You need to account for reward hacking and unexpected outcomes from training.
If I make a cake, and it unexpectedly explodes in the oven, it does not mean that the cake itself developed the free will to explode.
The model does not have free will because I did not predict what it would do!
alt Hacker News
If I make a cake, and it unexpectedly explodes in the oven, it does not mean that the cake itself developed the free will to explode.
The model does not have free will because I did not predict what it would do!