In the case of medicine it goes deeper than that, sometimes nobody really understands why a medication works.

This is also how most CEO live.

They ask their employees to build stuff, and have no understanding whatsoever of how any of it works.

A major school of German sociology places this at the core of its theory. According to this theory, modern societies are characterised in particular by the fact that individual subsystems of society reduce the complexity of their own (sub)system to the other systems to enable them to act at all; they fulfil what is called an "Entlastungsfunktion" (relief function). Prominent representatives of this school of thought are Max Weber, Arnold Gehlen and Niklas Luhmann.

In their view, it is modern institutions (public and private) which, as supra-individual entities, have long since become autonomous systems. The fact that the individual office-holders are human beings, meanwhile, is of little significance.

Hannah Arendt, in her theory of totalitarianism, attributed the effectiveness of both Nazi and Stalinist policies of extermination to the largly moral indifference of bureaucracy as a system.

In this sense, the task of controlling AI is a variation on the problem of harnessing a complex society consisting mainly of autonomous subsystems. This is a problem which has increasingly challenged humanity already for quite a long time. And it has been very difficult so far, even without AI ...

I'm going to say it's how everybody already lives, because while somebody may know some of those things there is nobody who knows them all.

Very interesting point. I think the counter-argument to that is that the complex modern society is based on somewhat “deterministic” systems, in that, even if a single decision or event isn’t rationally explainable in the moment, at least in the aftermath, it typically becomes understandable, maybe even reproducible. There is someone, somewhere, capable of explaining, maybe even multiple someones.

We don’t generally have that insurance with LLMs/AI, yet?

There are huge systems supporting that ignorance though. And where those systems are breaking down people do care.

the average person doesn’t have an internal monologue

can’t use a computer (they’ve had like 30 years now in first world developed countries)

many can’t even use their smart phone beyond calling, texting (many can’t type well), and doom scrolling (they get addicted to drugs, gambling, and other LCD activities)

many read at a 6th grade level. most can’t even calculate tip in their head.

meanwhile, the same smartphone can give them access to literally any information and knowledge the world in seconds. and now gemini can explain stuff since most ppl can barely read or think.

it’s sad out there.

but more importantly. it’s not my problem.