> Training data can't be the whole answer.

Absolutely correct. Anthropic showed that 250 examples can "poison" an LLM -- independent of LLM activation count.