It's already being trained on "public" (ethical or otherwise) data. So, it already has ingested that kind of "optimization" during pre-training and training.
I don't think you can fine-tune your way out of it.
This is far from widespread at the moment, so it'll be possible to at least use the current cutting-edge models locally in the future.
People still think these things are smart. That if their word generator eats enough of the Internet, it will somehow give them the real information that's otherwise hidden. Or perhaps a better word; filter the bullshit.
To filter bullshit it would first have to understand bullshit, and it doesn't. That's why an LLM will tell you the solution to a problem that doesn't work, and argue with you when you correct it.