> Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear

No it is not.

They are three places you can put things, amongst an infinite other number of places you can, but they are not some inherent core distinctions.

Is a javascript file code or data? If I read it to measure file size and don't execute it, is it ever code? If I read in a file and do different things in the code based on it, and never alter that file, how is that any different from having if (x) baked into the code? Why is it user supplied data if we read in a file and do things based on the contents yet code when I read in js and execute stuff in a VM?

They are just different places with different access controls that alter the outcome. There's no magical distinction between code, data and admin data or anything else other than really who gets to change what. And then there's obviously not just three named buckets of "stuff", it's as many as you want.

> I don't think the oft-repeated "AI is fallible just like humans are" applies here.

You're missing the point here, the point is we have built big things with fallible components. Therefore the idea that a component is fallible does not just from that fact mean we can't do useful things with it. And it's a very relevant point if we're looking at "where can we use this in place of humans" - if we had a human there before we were not starting from a perfectly reliable point.