alt Hacker NewsI quit OpenAI because its culture is broken
Comments
What about the sister rape thing?
Well and I didn't even started to work there. So I win this morale contest.
You quit because you got vested
Frontier AI labs are turning into the navy seals. Everybody who worked there will feel compelled to write a book about it
a safety alert isn't much of a control if it doesn't actually stop the system. i'd rather see proof the shutdown path works than another report saying risks were considered.
Can we take any of these "safety leaders" seriously though? I still remember "GPT2 is too dangerous to release".
I feel like we're too far into the crying wolf part. Basically none of the doom and gloom scenarios have come to pass. Instead, AI has gotten better at censoring itself.
The biggest AI safety risk is when an AI tells a police officer "he's the suspect" and the officer believes the AI without confirmation.
“I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems"
... over an unbounded timeframe?
And how exactly?
Those are very round numbers, but also very specific. Can we get some accounting on how you came to that? Anything? Vibes?
I mean, if you want me to take you seriously, let's have a deep discussion with things that can be measured. I absolutely agree that OpenAI and friends aren't being restrained enough and are acting with recklessness, but declarations of doom based on vibes isn't cutting it.
I use opus 5.5 and chatgpt to create PowerPoint, opus really follow the instruction and their PowerPoint generator really well, while chatgpt struggling to even create basic shapes.
I had this thought a while ago, the more complex the systems get, the more reliant on "smart" people powerful institutions will become
And the less their input is valued, the less that pool of smart individual will want to participate
So powerful institutions will end up relying more and more on having to trust these automated systems that they can't fully understand
It'll lead to a inevitable catastrophe, call it apocalypse if you will
> Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.
The problem is, we are running in a globalized world, and even if we were able to make our companies bend to our will - China does not give a shit about anything ever since the US kneecapped the WTO. And they will do anything to get an advantage over us.
We need a new rule that mandates every such "I am leaving <AI company> because of <concern>" post to disclose how much equity they have in the company and how much they have already cashed out. Easy to suddenly find a moral compass when you, your kids and their kids never have to worry about working for money again.
Mr Robinson if you are reading this – if you are truly concerned about AI safety share proof of donation of 100% of your OpenAI earnings and equity towards undoing the damage you have done to society during your time there.
In the absence of that this is simply a career pivot into being an AI "influencer" and/or raising money for a new scam.
why do I feel like these people are paid to quit as an inverted marketing stunt
This isn't new. Facebook has been screwing people over for a long time. AI is just the latest in a long stream of relentlessly exploitative, evil behaviors perpetrated by the same group of people.
“I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.”
There are a gargantuan number of extremely intelligent AI researchers, Turing Award winners, and the lab CEOs saying the same thing. They are the ones closest to understanding the technology.
Where there’s smoke there’s fire.
We need emergency laws to stop all AI development work immediately. It will take us decades to make sure this technology can be made safe as we only have one chance.
Is this news at this point?
You can basically time your openai releases by if another safety person has quit in protest
Idk why anyone thinks there can be AGI and alignment, seems almost like an oxymoron to me.
"I believe that we need to look deeper than specific rules or new laws. We need to talk about culture.”
lol. Please tell me some abstract concept like one employee's view of "culture" should be the priority over "rules and laws."
[flagged]
[flagged]
[flagged]
[flagged]
I think its pretty easy to solve these problems: Whenever an AI agent commits a crime, the CEO is held personally accountable, as if they'd committed it themselves.
[dead]
>“Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster,” he wrote.
This sounds more like an attempt at regulatory capture. Current AI systems aren't physical infrastructure that can just run away like a nuclear power plant, for example. At the end of the day, AI is still just software running on someone's hardware.
OpenAI is better off with less of these cultists around.
The LeCun article being posted at the same time as this is quite apt.
These "safety" people should have spent more time reading actual cybersecurity textbooks and less time reading EA forums and less wrong (or in Robinson's case, it appears, being policy wonks). Maybe then these labs wouldn't be totally incompetent.
The LLM cos leadership are all nutters
This smells more like guerilla advertising. These things are not getting more intelligent, they are still no smarter than a slime mold, we are just burning more power to make slim mold that eats tokens than yesterday
What I don't understand is what's the excuse for all the bad UIUX?
You try to go through files, and photos and the UI panics. You continue a conversation from your phone onto your computer and you lose part of the chat.
There are many more issues like this that are just so basic. You have bots that can attack governments but can't build a functional UI?
How many hours of ChatGPT does it take to implement a lock / consistency on a chat session so you don't overwrite it?
Buncha r*tards
I think David's expressed concerns are valid... there should be more safety engineering around LLM to avoid data loss and other accidental issues caused by LLM models.
And, I find the timing of "all of this" very interesting:
- we have a coordinated marketing campaign around discussions of AI safety. I am not saying David is part of that campaign but it would not surprise me. OpenAI and Anthropic are talking about safety and the need to slow down. People are "resigning". Techno-influencers are also being offered money to talk about AI safety, etc.... and some of them are taking that money and others are not.
- Cheaper models are keeping pace with Frontier models, albeit with a 3-6 month lag... which indicates that frontier labs have no moat unless one is created for them by law or some other market force.
- we have a ballooning debt crisis in LLM model hyperscale spending... revenue from serving LLM models is lagging the debt payments which are coming due by a wide margin. If something dramatic doesn't occur in the near- to mid-term, the financial bottom is going to fall out of this thing and the AI industry is going to suffer a major collapse of some sort.
The cynic in me almost feels like this is staged. An article about culture that is actually an article about how big and smart and scary AI is. I think Michael burry recently said something like “IPOs need hype, calling AI big and scary is hype” in reference to the anthropic IPO.
Last I checked it was still within the laws of physics to run air-gapped systems, and to ensure it is physically impossible for a model to “escape” or gain access to information it shouldn’t have. Maybe this safety guy should have been worried about that and not humble-bragging about writing 12 reports.