alt Hacker NewsI'm sure some people will have issue with my phrasing but, honest question:
Are there examples of where we have collective decided not to pursue knowledge? Successfully?
I guess nuclear weapons might be the best example though research doesn't seem have to actually "stopped" as much as gone underground and we still have country trying to climb that ladder.
But I don't know how relevant that is to LLMs/AI. It almost feels like pandora's box is open and our only option is continue to improve them. There is clearly value in what they do and while I can absolutely see the dangers, for example: authoritative governments and surveillance, I'm not convinced to throw the baby out with the bathwater.
All of technology back to the printing press (and probably before that) could also be said to make it easier for governments to oppress their citizens. Making laws (and enforcing them!) to prevent governments from doing these things feels like that route forward, not trying to stick our heads in the sand.
Perhaps I'm horribly naive, perhaps I just see the SciFi future I've spent my life reading and dreaming about on the horizon and I'm blinded by the reality, perhaps my ideals around "knowledge deserves to be free/accessible" are misguided. I don't know.
Replies
If you read the scenario, you'll see that the regulations are mostly about what people can do with giant compute clusters, and not about the ideas themselves. The ideas themselves are required to be totally transparent to the public.
As for historic precedents: Human cloning, human genome editing, and mirror life seem like one precedent; nuclear weapons and nuclear energy another; come to think of it I think drone delivery was strangled by regulations too...? Plan A isn't a proposal to never build superintelligence, it's a proposal to build it more cautiously and transparently.
No, of course not. That would be an insane trust fall. Even relatively small advances in technology give a country world dominating power. Fun fact: India was militarily superior to Britain in the 1600s—a gunpowder empire with a million soldiers—but was taken over by it in the 1700s. Britain’s edge was small: lighter, more maneuverable cannons, standardized ammunition, better military and political organization. Not a first world country versus a third world country—more like the dynamic US versus a sclerotic EU. And that modest edge led to 200 years of colonization.
If we slow down on ASI voluntarily we’d be allowing a gap to open up that would make the difference between colonial europe and colonized Asia/Africa look trivial. It would be insane.
Yes, there are examples of where we have collective decided not to pursue a particular technology tree.
For one, Japan banned guns for a few centuries. (Its warrior class was politically powerful and judged that guns would disrupt class relations too much.)
And there have been successful world-wide bans.
For example, following the invention of recombinant DNA technology, scientists convened the Asilomar Conference in 1975. They established a voluntary self-moratorium on certain types of genetic engineering until strict laboratory containment protocols were created.
In the 1980s, bioethicists, theologians, and researchers established a hard ethical line between somatic editing (treating an existing patient's non-reproductive cells) and germline editing (altering future generations).
No one has performed the latter form of genetic engineering except for Chinese scientist He Jiankui in 2018. (Chinese society used to be more ambivalent about the technology than the West is.) In response, Beijing heavily tightened its laws, classifying heritable gene editing as a high-risk medical technology subject to the penal code, and He Jiankui was sentenced to three years in prison.
Taking the approach with AI that we took with the atomic bomb would be catastrophic. If the only people who are allowed to use this technology are governments, intelligence agencies, and a select few anointed companies, then the risk of authoritarian misuse will skyrocket.
I worry that any attempt to limit their use and development will be abused and misdirected. We are already seeing people like Anthropic doing this, they are trying to use anti-AI sentiment to engage in regulatory capture. Go watch Dario’s speeches about how open weight models are dangerous and how they are “not really open”. Everyone can see that much of this “safety” conversation is ultimately just a tactic to shut potential competitors out of the market and establish a monopoly/duopoly.
The decision to not go with the development of extremely large thermonuclear weapons might count - the US Sundial Project was supposed to be about 10 gigatons of TNT. Not the most practical weapons but once you get to a certain size delivery arguably stops being a problem - its going to kill everyone anyway so doesn't matter where you let it off!
https://en.wikipedia.org/wiki/Sundial_(weapon)
Edit: Mind you, I wonder if the design for Sundial is stored somewhere...