logoalt Hacker News

Silhouettetoday at 10:38 AM0 repliesview on HN

My point is that an LLM can't attempt to connect to anything by itself. All an LLM does is produce a stream of output tokens - and that was already quite useful as a coding aid.

It is the harnesses that some people are now wrapping around LLMs to interpret the output from a model as commands to run (or other executable instructions) that are creating all these new risks. Remember that this is still a very recent development and still more recently amplified by the use of feedback loops and long-running agents intended to operate with minimal human supervision.

It is going to be increasingly important to understand exactly what these tools are doing and why for both correctness and security reasons. Not conflating their capabilities with the underlying model that purely generates data is pretty fundamental here.