Eh, this has been a disappointing series to read from the person that wrote all those Jepsen write-ups and the technical interview series. All articles are mostly a regurgitation of all the negativity that gets aired here all the time (a lot of it already fixed or debunked) and 0 discussion of utility. This is more befitting Zitron than aphyr. Where's the sharp incisive wit and deep insights?

At least he realizes this technology is unlikely to slow down. With international relations as they are, it's MAD all over again, only the "D" is a fuzzy, hypothetical thing nobody can name, so even that bit of deterrence is lost. Yet finally he ends with the most uninspired advice of all: "we should try, unsuccessfully, to stop it."

Everyone must understand: for all of history, progress and productivity and value creation overall could only scale with people. Now it can scale with power and compute. This is a tremendous economic force, akin to a force of nature, that is nigh impossible to stop. (I always did think the Butlerian Jihad was the biggest plot hole in Dune.)

My advice is this: we have no choice but to adapt. We must realize that, by a stroke of luck, this is a power available to us more than the capital class. If they can scale without people, so can we. But because harnessing AI effectively requires hard skills -- at least for now -- that the capital class don't have and used to pay us for, we might even scale better than them!

Carpe diem.