alt Hacker NewsI had a long ranting comment I deleted. I just don't like this trend of people presenting work in a way that makes you think some combo of 1) they discovered from scratch themselves 2) it's new 3) they didn't try to cite or acknowledge where they learned it/point to good sources 4) they don't really care about trying to teach something deeply, they want shiny stuff that makes them seem deep.
This post references specific parts/calculations, but you'd never know it was not news if you didn't know better.
Replies
The post says this is all part of gzip and LLMs, what are you saying? I’ve been using gzip my entire life. I read between the lines “this is common knowledge” throughout the piece. Throwing in some names and dates only makes this super clear story harder to read (and more like studying then the playful exploration this post was intended as).
I'm of two minds here. The pro is that the "you could have invented this" walkthrough from first principles is more engaging than "and then so and so introduced this term in 1972 and the definition is such and such". This style is a reaction to that boring and dry teaching style and tries to push towards what eg Feynman pointed at in the Brazil critique.
The con is that you don't get to understand and see any of the history of the ideas or even the ballpark when it was discovered, you attribute it to the blog mentally and you don't know what is how new or old and can't reference it properly when talking to others.
You're reading this the wrong way I think, citations aren't given because its obviously a pedagogical article about well established stuff. Much like you wouldn't give citations in a blog post explaining calculus.
they discovered from scratch themselves
If you followed the data compression scene in the 80s and early 90s, there were plenty of reinventions of LZ-ish and Huffman-ish algorithms (I also coded my own variant...), and people even tried to patent some of them, so at least for the basics I think it is something that many can discover independently; of course in these times, it's more likely they didn't.
The first sentence says she came across it when reading about compression. I didn't read that as her claiming to have discovered the idea or that it was a new idea, I read it as "today I learned". I think somebody who was unfamiliar with how compression algorithms or language models work would find this an approachable and interesting introduction. Not everybody studied information theory.
Perhaps tangential to your point, but I often write blog posts (although finish and publish far fewer than I start) where I write about something as it has occurred to me, informed by things I've absorbed no doubt, but without specific research. In such cases I explicitly avoid searching out prior work as a) seeing that something is well discussed and explored can take away the motivation to explore (in the same way reading puzzle solutions before starting might), and b) to avoid having green shoots of ideas shaped by the current of existing consensus. Now that doesn't mean I don't come back after doing my own thinking to see what the more well developed literature of people cleverer than me, who've thought far longer than me think; I just don't want to snuff out my own exploration at the start.
As I say most of these I never publish as I'm mainly using writing as a vehicle for thought, but when I do I'm never sure how to flag them. I don't want (imaginary, lets be honest) readers thinking I'm deluded into thinking I've found something new. I want to come up with a tag I can put on them which adds a pithy disclaimer card at the top or something so I feel more comfortable publishing them.
Does everything have to be "news"?
I think you'll find that 95% of all academic presentations are telling stories out of other peoples work.
I'm glad to see someone feels similarly. There is nothing wrong with ignorance, but there's no excuse mistaking learning for invention. Especially from someone bearing the title "Developer Educator"
I don't think it's the case here, but worth noting too that LLM-written blog posts adopt this tone seemingly by default.
Never the least bit of surprise, wonder, doubt, or frustration to get in the way of the steady staccato beat of metaphors, conclusions... and three-item lists.
And yet to this day, in AI threads, so many people act shocked and surprised if you dare follow the obvious implication and claim that understanding is a form of lossy compression.
Reminds me of Stephen Wolfram "discovering" things in the sense that other people would say "today I learnt".
I'm glad you're pointing this out because not only are these old insights, but I'm also pretty sure I've seen variations of this blog post years ago on even HN already.
The author acting as if they discovered this independently had me feel the exact same way. Kinda irritating and almost ... disrespectful? Not sure of the right words to describe it tbh
I don't think this is a fair critique.
The author of the post uses standard terminology like entropy coding and arithmetic coding, and cited a paper "in 2023, Google DeepMind released a paper arguing that language modeling and compression are two views of the same thing" which discusses it further.
This blog post is great. Well explained, and clearly took a lot of effort.
I don't interpret it as them claiming to have to discovered it independently.