Kinda weird to generalize "LLM". Every lab, every model is different. Has its own biases, reward functions etc.