Not sure where I implied they are "raw base models" and not sure what "various adjustments" means here or how "ton of RLHF" contradicts anything. If we look at research for open source models, "adjustments" usually come in the form of efficiency gains which directly contributes to the ability to scale or synthetic data pipelines to increase the dataset and increasing the context window.

rlhf = reinforcement learning from human feedback

(had to look it up)

More RLHF is in fact scaling.

Not just RLHF but also RLVR, and isn't that the litter lesson though?

My sense of the Sutton Dwarkesh interview was that he was calling out that he didn't mean just longer datasets, but rather learning through exploration and that's exactly RL.

RLHF is an increasingly small part of training though? From what I understand most of the capability gain is in RLVR

Nah, the last few generations have more RLVR in the data mix. Which is more CPU intensive and very much amenable to the bitter lesson as you can reduce the loss by doing more rollouts in your tool environment.

The scaling with reasoning models is more and more with things like verifiable rewards (coding and math), in line with bitter lesson and also Sutton invented lots of modern RL.