logoalt Hacker News

GMoromisatoyesterday at 5:02 PM2 repliesview on HN

Agreed--and maybe they should have fixed it.

But sometimes the "provenance" of the data is important. I want to know whether I'm getting data straight from some source (even with errors) rather than having some intermediary make fixes that I don't know about.

For example, in the case where maybe they flipped the latitude and longitude, I don't want them to just automatically "fix" the data (especially not without disclosing that).

What they need to do is verify the outliers with the original gas station and fix the data from the source. But that's much more expensive.


Replies

chapsyesterday at 5:10 PM

Exactly. This is a big problem with "open data". A lot goes into cleaning it up to make it publishable, which often includes removing data so that the public "doesn't get confused". Now I have to spend months and months fighting FOIA fights to get the original raw, messy data because someone , somewhere had opinions on what "clean data" is. I'll pass -- give me the raw, messy data.

show 1 reply
hermitcrabyesterday at 5:11 PM

Or just omit the rows that are obviously wrong (and document the fact).

show 3 replies