alt Hacker News> Any shared sense of rigour is just completely torpedoed by the LLM world, particularly the cloud LLM world it seems, and we are reduced to cargo culting. Nobody is any more right or wrong than anyone else.
There was always some of this in the tech world, long before LLMs came along.
I've sat in so many meetings when decisions were made based on "that's what _slightly more prestigious company_ does" rather than objective measurable criteria. (And the evidence that the thing in question wasn't universally followed by _slightly more prestigious company_ carried surprisingly little weight).
Absolutely I agree there has always been some cargo culting going on; that's true of all process-oriented businesses.
But people are now individually acting this way on their desks on an hour by hour basis. LLMs make cargo-culting inevitable because they are inscrutable and opaque.
There is always this sense in the LLM-proponent world that LLMs are at any moment as bad as they are ever going to be; line goes up.
But it seems clear that the gap between perceived and measurable productivity is still likely spent in poking entrails with a stick.
We are so used to probabilistic tools that have significant setup time before they become valuable and save us loads of time that we're at risk of repeatedly writing off that setup time without seeing the rewards, believing that one day it will actually work out that way.
(Which is most recognisable from the early JS frontend frameworks era.)
Meantime here we have an article that shows that a thing (longer context windows) that people thought would functionally solve a problem so we would get the value from all that setup does not, in fact, very meaningfully kick it down the road, and the comments are still full of entrails-and-stick work.