alt Hacker News> So we still don't have a reliable way to separate instructions from data when talking to an LLM
Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.
Replies
hypeatei • today at 1:51 PM
> Humans also do not know how to do this reliably
These are machines, not humans, so I don't understand the comparison. The point of tech advancement is that we eliminate entire classes of errors that humans make. You'd probably look at me funny if I wrote a production application that failed randomly in unexpected ways like corrupting data, opening security holes, etc. then explained it away with "well, humans do it too!"
➕ show 1 reply
I think the Stroop effect ("read these colour names, each written in a different colour") is probably the purest demonstration of this. Humans are trivially prompt-injectable.