The timeline is very confused, but the arc is mostly right.
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
> Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando")
I’ll die on this hill. You should pin your dependencies so you don’t get pwned, and then use dependencies for everything mildly complicated that isn’t core to your product.
I bet 90% of people reading this work somewhere that has built a model router in the last year.
I understand the sentiment. You work at a sparkling water company, and building a durable workflow engine is much more interesting than building a website.
But without that tech being your core product, maintaining and evolving it will lose priority. If it requires barely no maintenance, it’s even worse, because when it inevitably does, you don’t know if deploying a new version still works.
Use stuff that exists. If the dependencies don’t do what you need, fork them, and consider contributing a patch.
Powerful. I feel like this is emblematic of the economy as a whole since the 80s. Consolidation and outsourcing makes hyper-efficient but brittle businesses.
> Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers
To me this is the crux of the problem. The issue is not that a particular company’s services went down, but that 25% of the world was relying on it being up.
And now add vibe slop onto that, in both the dependencies and the implementations and the infrastructure management, and you really have a recipe for cascading failures from which it’s difficult to recover.
I’d like to add that the perpetual gold rush during this period has created a culture of self-serving that few other professions can compete with - basically institutionalised low-level corruption.
Decisions are made on the basis of trend chasing, what’s fun, and what’s good for the devs’ resume. And it’s completely accepted, hence the world is overflowing with Rube-Goldberg machines that basically defrauded whoever paid for it. Try doing that as a carpenter and see how long you’d stay in business.
> Which is great for corporate accountability
This is a common take, but it's most attractive for its cynicism, it doesn't fully represent the reality.
Cloud vendors are also great for allowing corporations to implement systems that would be prohibitive for them to implement, operate, and maintain as reliably on their own. Even big enterprises often struggle to implement everything they need, that a cloud provider can provide out of the box.
I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off. There were plenty of warning signs, but the final straw came when there was a major security breach, which was unacceptable because this company had federal customers and were subject to all sort of federal regulations, including CFIUS because the parent company was non-US. (They once had to rewrite an entire major system from scratch because of a breach of CFIUS regulations that made the government consider their code to be potentially compromised.)
That may be an unusual case, but the point is that if even large, well-funded, established companies can get a lot of benefit from cloud systems, that goes double or more for smaller companies, companies in less attractive industries, etc. etc.
(Pointing this out sometimes triggers people who maintain servers themselves, especially here on HN, but the point is those people don't work at the companies in question, and essentially don't scale - i.e. it's difficult to recruit enough of them and manage them and put together a durable system with enterprise-grade reliability that depends on having 10x sysadmins.)
And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
> Anxious and lazy dependency-first development culture
I think this is the completely wrong takeaway.
Ober the last ~15 years I have probably used around ~10 web frameworks/libraries
_not a single one was even close to being complete or mostly correct, even just for standard HTTP ~1.1 handling_!!! (The routing, path, header handling. Not any of the complex stuff in HTTP2, Quick etc.)
the problem isn't using dependencies, that is the solution (not reinventing the wheel again).
And preferring reusing solutions which are more complete, practical testet and have more resources to maintain/improve basic functionality is (in most situations) not "anxious" development, but (should be) common sense.
The amount of hidden complexity in even somewhat basic stuff like HTTP is enourmous, rewriting everything all the time is wasting billions across the industry all the time and (in general) increases the chance for bugs.
The problem isn't "using dependencies" or small dependencies, but:
- availability (or lack) of high quality well maintained dependencies
- "no one" wanting to pay for software dependencies
- misguided tendencies for 'clever' & overengineered solution in one extreme (e.g. the stuff which lead to log4shell)
- misguided obsession with deceptive simplicity in artificial examples which lead to 1001 incomplete solutions because of the obsessiv simplicity doesn't stop at cutting out over-engineering but also cuts out fundamental parts
- lack of widely adopted, effective, tooling & infrastructure to trust dependencies.
- lack of tooling to more efficient reuse and compose working parts, the way we do programming seems just very fundamental broken.
- also a lot of legacy tech we are stuck with making many things unnecessary complicated
Also as a side note, number of packages != number of entities to trust. Weather you use 10 or 1 package from the same source, maintained and published in the same way often makes little to no difference for supply chain security.