alt Hacker NewsWhat you’re saying is correct if the model is trained with all the knowledge humanity had, has and ever produce. But at the moment the next token prediction is quite limited to the training data.
Things could change if the model supports re-inforced leaning. That way the LLM would change the weights in real time based on a feedback loop, but again that could vastly improve the quality of the token prediction or completely degrade it as well
The distinction I would make here is that computer code is logical transformations on arbitrary data, not the actual data itself. An LLM can learn the entire space of logical transformation patterns from existing code, and can hallucinate new logical transformations, using a computer as a validator for the logic, so an LLM can create new logic as well as repeat existing patterns, and that logic can be applied to novel input data that the LLM has never seen before.