alt Hacker NewsWell, what do you expect? LLMs are trained on blithering, mostly from web sites. So you get blithering out.
There's an important point in the article, that forcing a style onto an LLM is lossy. Although he doesn't seem to mention it, forcing a style may result in the insertion of new blithering, possibly made up as a hallucination.
Replies
I think that was a good enough explanation for gpt3.5 -- these days, labs are extremely capable of post-training phases that eclipse that kind of training phase -- and hence of choosing whatever style or tone they wish.
eg., OpenAI has gone a long way to making reasoning token-efficient by having reasoning piovot off terse langauge -- whereas anthropic appears to be doing the opposite.
> Well, what do you expect? LLMs are trained on blithering, mostly from web sites. So you get blithering out.
AI labs can now ask the LLM to translate and filter the data, to create new training data that makes more sense and has better style.
is it lossy though? That didn't make sense to me. You can tell it to use Simplified Technical Language and also still have it give you all the detail. it's just another piece of the prompt that produces the output. it's not like there's "pure" llm output and then "lossy" output guided by a prompt.
> Well, what do you expect? LLMs are trained on blithering, mostly from web sites. So you get blithering out.
I realize this isn't entirely serious, but I can't resist pointing out that this doesn't seem to be a good explanation for why LLMs write the way they do. When we've experimented with LLM writing style on open-weights models where you can get a base model (pretraining on text only) and an instruction-tuned variant (pretraining + post-training with RLHF and whatever other human-evaluated tasks), it's the instruction-tuned variant that shows the weird writing quirks. That is, the writing style is not because of the training texts, but because of whatever tasks the LLM companies do in instruction tuning. https://arxiv.org/abs/2410.16107
I'd speculate that this is partly impressed human preferences (the human raters unintentionally reward a particular writing style) and partly because of the chosen tasks: they're training the LLM to be good at, say, summarizing text, so it develops a style that's good at being informationally dense.
At any rate I've seen this same phenomenon with Llama and Gemma, and will be trying soon with Qwen. Unfortunately none of the commercial models lets you access the base model, as far as I know.