alt Hacker NewsThis opens up so many interesting opportunities. In principle one could compile a small VM inside the LLM weights. The VM can host for example a Python interpreter, a shell and tools like sed, cat, grep etc. Instead of LLMs having to do slow autoregressive token emission with the current "external" tool calling, it could literally execute everything except web search, inside the LLM, with just matmul operations. You could probably optimise this so it runs in batches. You could have a gdb like application that can step through the LLM forward pass. Anyway, just a crazy thought. Have a great weekend everyone!
Replies
I wonder if you could train an LLM while holding the weights implementing fixed-function regions constant, so the training would produce a model that natively knows how to use these tools inside it's own reasoning steps.
I suppose it doesn't need to necessarily be in the weights, a hybrid LLM + internal fixed function blocks that aren't encoded in the weights could be quite interesting.
Combine this with latent reasoning and it gets pretty wild.
A very useful fixed function region in my mind is a symbolic execution engine, capable of only following a well-defined grammar. Perhaps a lisp would be good too?
Totally. It's like how we put brainfuck interpreters everywhere.
You can just do that anyways.
You might like this: https://research.google/pubs/latent-reasoning-with-supervise...