alt Hacker NewsWe are already on the cusp of fully automated reasoning, and once we have fully automated reasoning, OpenAI and Anthropic can just dedicate part of their compute towards generating new high quality novel output, which will then be fed as training data during pretraining of subsequent models.
Replies
fatata123 • today at 1:43 PM
[dead]
qsera • today at 11:37 AM
That is like saying we can get unlimited data compression by feeding the output of a data compressing program into its own input..
➕ show 1 reply
I don't believe that to be possible in general. Because we've already had Millenia of philosophers attempting to make discoveries through sheer reasoning and with the small in the grand scheme of things exception of formal logic failed to do so. Which leads me to a principle: No matter how smart you are, you still need the real world as a reference.
Once again LLMs will have to be bound to a source of entropy or feedback of some sort as a limit. Sure you might be able to throw terawatts of cycles at say music production but without examples of what people already like or test audiences you cannot answer the question of whether it is any good.