logoalt Hacker News

ghoblinyesterday at 2:12 PM1 replyview on HN

There’s a difference between "safety matters" and “safety is the primary constraint". Most companies manage risk to an acceptable level while optimizing for speed and cost. Aerospace companies optimize for minimizing catastrophic failure, even at extreme expense. Treating a potential GDPR fine as equivalent to a flight-control failure ignores that society, regulators, and markets treat those risks very differently. The inconvenience and economic cost of your Discord messages leaking is not the same category of harm as your pacemaker controller failing. And because the majority of economic activity sits in that lower-criticality category, it would not be surprising if highly specialized, safety-critical human software engineering becomes more of a niche, while much of routine software development becomes increasingly automated or commoditized.


Replies

swiftcoderyesterday at 2:32 PM

> Treating a potential GDPR fine as equivalent to a flight-control failure ignores that society, regulators, and markets treat those risks very differently

Agreed, though I think that if GDPR fines were actually being levied at the recommended 4% of global revenue, we'd start treating them more similarly to a 737 crash.

> The inconvenience and economic cost of your Discord messages leaking is not the same category of harm as your pacemaker controller failing

Sort of depends who they leak to. Your teen classmates who bully you to suicide? Your abusive ex who is trying to track you down to kill you? The 3-letter agency who is trying to rendition your family to an internment camp?

There are a lot of seemingly benign failure modes that become extremely lethal given the right circumstances. And because we acknowledge the potential lethality of something like a pacemaker failure, we have massive infrastructure dedicated to their mitigation (EMT teams, emergency external pacemakers, surgical teams who can rapidly place new leads, etc). For things society judges less important, mitigations are often few and far between