alt Hacker NewsThis is the only statement I disagree with:
> PowerShell gets a lot right with structured data.
CLI programs should operate on text. If you want to parse and format it, do so, but the default output mode should be plain text, so that I can pipe it into grep or awk without a second thought.
I am continuously irritated that the AWS CLI defaults to outputting in JSON. No one (I hope…) is using that tool in programs; that’s what boto3 and its ilk are for. But if humans are reading it, why default to something that they’re almost certainly going to be piping into jq if only for the formatting help?
Replies
I always hated powershell for the same reason, and then I read a really long retro by the guy who ran the project and it makes sense now.
Basically Unix has a long tradition of "everything is a file" and a big ecosystem of coreutils that are based around text and windows.. didn't. You can't look at /dev or /etc and learn anything about the machine. They had a few generations of APIs and wanted to give admins and power users any shell at all instead of a GUI. So the shell is centered around making those APIs accessible, rather than piping grep and sed or whatever.
Powershell commands automatically format the data for you, you can pipe it to grep just fine (I do it all the time). It's just that you can also access it in a structured way if you need to. It's the best of both worlds.
Linux tools that are starting to output raw JSON by default are indeed a nuisance, but how else can you achieve structured output if no standard shell supports it? It's a chicken and egg problem.
I'm not sure I agree.
In the AWS case the tools talk to an API server, so sure, you can call the API server directly or use a wrapper that does, but what about all the other CLI apps that don't? The CLI program is the API.
I built a CLI program that wraps luks + btrfs, and they only offer a `--json` output option for a few commands. I have to write an ad hoc parser for each command since raw text includes arbitrary formatting and presentation lipstick that the creator came up with. And I have to do extra work to avoid breaking changes at the parser level instead of the higher data level.
If I had to pick between the two, json would at least solve the data representation part so that I can build on top of it. And it's trivial to go data (json) -> pretty print rather than pretty print -> data.
I can see it being annoying if all you care about is using CLI programs by hand, but it seems like a mild upside compared to the downsides the second you want to consume it programmatically, even if it's with a chain of awk, cut, and tr.
Slightly off topic, but it is worth being familiar with AWSCLI’s built in jmespath JSON transformation system. That saves on the need for jq in a lot of cases. Given that awscli bundles that, I generally forgive it for defaulting to JSON (especially given how deeply nested and interlinked many AWS API results are), but reasonable minds can differ. AWSCLI’s schizophrenic use (or not) of pagers for tiny results, on the other hand, is less defensible.
I thought I would like Powershell but it was another big disappointment for me.
It's got a lot of the unpleasant clunkiness that something like the Bourne shell owes to decades of compatibility, but Microsoft doesn't have that excuse. Despite this, it's gratuitously incompatible with itself, I had code which worked fine, we upgraded Powershell oh, now that won't work, just fix all the scripts. Crazy. It clearly wants an NPM-style experience where you seamlessly incorporate other people's work, but then it doesn't really deliver this well and so you often end up manually copy-pasting.
If Powershell was a one man project and this was their Beta I'd say it is promising. But it's a project from the Microsoft corporation for 20 years. Do better.
I think you're mistaking text-with-structured data for structured data itself.
Because unix shell is irrevocably text-oriented, kludging in something like JSON is basically the best that can be done when you start to want to do structured operations on structured data. (I'm sympathetic to your point about the AWS CLI tools doing JSON by default though--that just sounds like bad design.)
Being text-oriented imposes drastic limits on composability. Because there is no structure, every element of a pipeline needs to do its own parsing of the input data. This leads to brittle pipelines where every element is tightly coupled to its input's textual representation.
As an exercise, try to write a pipeline that sorts podman images by size without removing the column headers[0]:
As far as I can tell, there is no way to do this in a manner that's even remotely composable. Your best bet is to basically do everything from within awk. Whatever the result would be, it certainly won't be pretty!Contrast that with what you can do in PowerShell. You can write a couple of standalone functions[0] that are readable and composable, resulting in this pipeline:
[0] Repurposing this from a blog post I wrote: https://www.cgl.sh/blog/posts/sh.html#this-should-be-basic