Confidently guessing, and building an entirely new set of assumptions based on a confident guess is the single source of the most frustration I experience using these. "Intelligence" does not look like that, and all the prompt hacking or clever hooks in the world doesn't seem to stop them from doing it.
I don't know there is really a solution. Some models get better at this for a while, then regress. It's like whack a mole. Until this is 'solved' these will never be fully human out of the loop, but I am suspecting this is a fundamental nature kind of thing with them.