alt Hacker NewsI don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further.
And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators.
This isn’t like niche, tin foil hat stuff either. People have been writing, singing, making blockbuster movies about every aspect of what’s going on right now, edit: for decades.
We all know, but somehow we don’t, OpenAI autonomously hacking into another company should have counted for something, but I guess not. Anyone else feel like they’re taking crazy pills? I could make a comedy about everything going down, and the unshakable complacency of people
Replies
Follow the money: who told you that OpenAI's models autonomously coordinated to hack external systems? What incentives might they have to want you to believe that story? Are there priors which demonstrate them benefiting from telling similar stories, regardless of their factuality?
But to your counterpoint, let's say the story is 100% true, because I agree it is at least plausible. What would the incentive be for the HN audience to believe it? What priors might support their disbelief? For my part, I don't think it's because people lack imagination. I think it's quite rational to question the authenticity and impact of the claims being made. What's worse, believing the story and being wrong, or not believing the story and being wrong?
That said, I agree with you: the impact we're having by not changing course is quite dangerous, the scale is dangerous, the inability to reverse the harms is dangerous, and the lack of collective effort to regulate further damage is dangerous.
It's true, humans are dangerous when trillions are involved. See: climate change.
Trust, or lack thereof. People don't trust OpenAI, a company whose very name is essentially a deception and a lie. People don't trust the tech industry in general anymore. Most tech companies act as a tax on otherwise productive business. AI companies and their leaders rose money by going in front of the public and saying "These things are extremely dangerous. Let us study them to mitigate the danger." And now they want to collect hundreds of billions in revenue. So yeah people don't trust what OpenAI has to say. They were supposed to mitigate this outcome from happening in the first place and instead they have accelerated it.
It's not that dangerous, OpenAI just shit the bed building their infra. Write safer software and you'll be okay.
Remember when gpt2 was too dangerous to release?
Something being dangerous and sama saying something is dangerous are not necessarily the same thing. Especially when he’s got everything riding on this bet
It's not dangerous to go further unless you're prepping for IPO at anthropic or openai.
Open weight Chinese models are basically matching state of the art closed models at a fraction of the inference and training costs, which puts a hard cap on OpenAI's future inference margins.
They're not going to get any sort of multiplier if they keep paying to train models, so they're trying to ban model training.
It won't work long term, but it could totally screw over the US for the next decade or so. Even worse than the economic issue: Consider the implications of "alignment" succeeding. Alignment to whose values? The pedophile-felon in chief? Even worse, tech CEOs?
It's dark times when China's basically our last best defense against totalitarianism.
There are smart, non-AI people who are paying attention to this field, and they are ringing the alarm bells.
Whether we listen is another matter.
I blogged about this recently:
> And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators.
Cause data centers are causing real immediate damage by people who hope to cause a lot more scifi level of damage later.
> OpenAI autonomously hacking into another company should have counted for something
It totally should. That company is negligent. I dont worry about models tho, I worry about damage Alman and his people will cause.
Based on your replies in this thread you seem to have only superficial knowledge about how machine learning and LLMs work. I strongly recommend you invest some time in learning how LLMs are built and function. If you truly think this is apocalyptic isn't it a good idea to understand what you're up against?
… or the safety argument is an attempt at regulatory capture and an effort to outlaw open models.
The absolute nightmare scenario for these people isn’t terminators. They’re fine with that, and in some cases are already doing it or supporting politicians who are doing it. Autonomous “kill chains” are a thing. It’s just happening overseas… so far. The politicians doing these things were backed by the heads of these companies. They don’t care about AI killing people.
No, the nightmare scenario for these guys is there is no moat. Their whole empires, which are built on training models on open source and sometimes pirated data, are easily duplicated. Worse, recent progress on models at the 30B size suggests that large gains in efficiency or compression are on the table. That means someone might release a cheap to run frontier grade model… or someone might crack distributed continuous training.
In other words… there is no moat.
So they need to scare some politicians into heavily regulating the space before that happens.
> because it’s literally getting dangerous to go further
What if there is no further at all?
You need to stop being so credulous especially regarding an individual that has spent his entire career deceiving others for monetary gain (also their deeply anti-human beliefs).
https://en.wikipedia.org/wiki/Don%27t_Look_Up
A movie fit for our time.
You can produce detailed descriptions of the incident, verified by adversarial parties, and some people will still scream "it's a conspiracy! It's a marketing stunt!"
This is all very unfortunate--there's a meaningful chance that AI will cause unprecedented disaster, with the HF incident being just a small preview, but people would rather squawk "stochastic parrot" for the millionth time than revise their beliefs.
We collectively accepted that we don’t care when we chose not to adopt memory-safe languages over the past decade+.
The only difference now is that the resources to find the exploits are being commoditized.
>People have been writing, singing, making blockbuster movies about every aspect of what’s going on right now, edit: for decades.
Theres Hyperbole and then theres whatever this is.
decades
https://en.wikipedia.org/wiki/Darwin_among_the_Machines Samuel Butler 13 June 1863
There is an entire big world outside of Silicon Valley cults where literally no one gives a shit about AI prophecies. Shocking.
It is extraordinarily emotionally hard for someone to stare down the terrible implications of what is unfolding. All manner of rationalization and cope will be applied to come up with excuses; motivated reasoning.
The CIA director will call AGI capabilities "digital nuclear weapons" and Geoffrey Hinton will estimate a 50% probability AGI ends humanity, and half of HN will call every new evidence of disaster a marketing stunt.
[dead]
You must be really desperate if you resort to panic attacks like this.
If these models are so dangerous, then why hasn't OAI or Anthropic shown them dangerously escaping sandboxes, nefariously coordinating with other escaped AIs, and skillfully hiding from human detection *in public* with full logs shared where we can all see exactly how dangerous they are or aren't?
Right now the entire chicken-little-sky-is-falling argument is based entirely on statements from OAI and Anthropic themselves. These are historically conflicted companies who desperately need regulation to put the competition into stasis.
At least chicken little didn't have a bunch of devious CEOs with trillion dollar IPOs that depended on us all believing the sky is falling.
>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff.
we don't all buy everything sama says as factual.
>We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further.
the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regardless of truth or not these kind of steps have jaded people.
my read : "We are doing poorly in financials so we'll give ourselves a bit of breathing room and a momentum shove by claiming our work is so advanced that it's dangerous while simultaneously spinning down expenses."
<jon lovitz : "Yeah, too dangerous, yeahh -- that's the ticket.">