alt Hacker NewsYeah, that point was weak. The argument went from "Unix pipelines are not simple" to "unix based operating systems don't have an equivalent to Clojure's frequency function by default so it's worse"
OK but how would it look like if you had such a program? Shells are not known for having the extensive set of functions that real programming languages have.
Replies
ChrisSD • yesterday at 9:41 PM
I think the point was more that it's hard to compose shell utilities to make something new that's not supported out of the box. Instead you do have to write an entirely new utility and use that.
That said having key/value semantics and not just stream of bytes would make the shell much more versatile, at the cost of making it bigger.
➕ show 1 reply
I think the article is a bit weak when it's making this point, because it's not about Unix pipelines. It's about POSIX shell utilities being too bare-bones.
But Unix pipelines are not simple too. They have a couple of nitty-gritty details that often come out and bite you.
1. They can only stream raw bytes, so all the programs that deal with lists like sort and uniq have to separate items using a delimiter (usually newline). If you want to process data with that delimiter in it, you're in for a ride. And if you want to write a custom tool, you have to do all the splitting yourselves (luckily it's so common most programming language will provide a ready-made facility for you to do that). This is New Jersey approach again: "I'll make my code (the OS, the shell) easier to write, and in return make life harder for my users (the tool writers)".
2. Error are hidden by default in shell. Nowadays you can explicitly change this behavior, but you have to remember to do `set -o pipefail` and I don't think it was always there.
3. There is no data typing at all. Everything is binary or text. Nowadays a lot of programs just output JSON, and the users (if they even stay inside the shell) almost always reach for jq to parse it. But jq is not a Unix philosophy program: it's an entire streaming functional programing that can do quite a lot. But even jq gets hairy when you have to do a bigger query or transformation. In that case users often reach out to Python or another language and just move the business logic there.
I think this fits well with what the article is trying to say: Unix pipes are pretty easy (small) to implement on your own (compare that to something like Nushell's pipes). But the moment you to do something that's a little different than the happy path it was built for, you need to go for another tool (jq) that has its own built-in pipe and small programming language, because Unix pipes won't cut it. And for more complex (hehe) things, you'll have to reach for a larger (and simpler) tool: a full-fledged programming language.