alt Hacker NewsI don't know anything about this specific LLM thing but if it correctly uses the Nash bargaining optimiser then that won't happen.
This thing you point out is exactly why Nash demanded invariance under affine transformations in his solution. Using completely arbitrary units if I rank everything as having importance 1 million, that's exactly the same as ranking everything as having importance 1, and also the same as ranking everything as having importance 0.
The solution is only sensitive to diffences in the unitity function, not the actual values of the function. If you want to weight something very strongly in the Nash version of the game you also have to weight other things correspondingly weakly.
You are correct that Nash should address this because only the relative utilities matter, not absolute.
There is the potential for parties to get better deals by overstating their BATNAs, but then they risk the other party rejecting the agreement when a mutually beneficial agreement was possible - so it's not in their interests to mislead the system.