alt Hacker News> There are only so many safety-first companies and products
There are only so many companies that think of themselves as safety-first. In practice, basically all companies work on things that should be safety-first.
Does your software store user data? Congrats, you are now on the hook for GDPR and a bunch of similar data handling regulations.
Does your software include a messaging component? You are now responsible for moderating abusive actors in your chat.
Does your software allow users to upload images? Now you are a potential distribution vector for CSAM.
And so on... safety isn't just for things which can cause immediate death and dismemberment
There’s a difference between "safety matters" and “safety is the primary constraint". Most companies manage risk to an acceptable level while optimizing for speed and cost. Aerospace companies optimize for minimizing catastrophic failure, even at extreme expense. Treating a potential GDPR fine as equivalent to a flight-control failure ignores that society, regulators, and markets treat those risks very differently. The inconvenience and economic cost of your Discord messages leaking is not the same category of harm as your pacemaker controller failing. And because the majority of economic activity sits in that lower-criticality category, it would not be surprising if highly specialized, safety-critical human software engineering becomes more of a niche, while much of routine software development becomes increasingly automated or commoditized.