alt Hacker NewsIsn't all of what you described what post-training/RLHF is supposed to do? The internet is full of racism, so if you're just predicting the next token based on training data, you'll get racism (eg. Microsoft Tay), but that's more or less solved by AI companies now.
The post training is meant to make it more steerable (usually). I might not want it to write tests. I might not have a dev environment set up for an agent to run tests in it's loop. A major goal in post training is to make it follow instructions, which doesn't have to mean have particular instincts for code organization