> The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.

Funny thing, right around the beginning of the big AI takeoff, all the AI ethics people who visibly thought their job was something other than marketing got driven out of the big firms, and often the industry entirely, because they were in the way.

So, if too many think their job is marketing a few years later, well, there is a reason for that.

> if the model requires retraining for ethics reasons.

Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from?

> The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.

Or it's exactly those people who will be left. We will read about it in your book.

I have a perspective on this that will read as unkind to these people, but that's not my intention. I respect and even admire what they're trying to do. I just think the effort is maladaptive and misplaced.

Without directly touching one of the many, many third rails that are present here, I'd like to present this section from Google's Gemma paper on how they did their CBRN review;

    > In addition to our internal evaluations described above (Section 5.7) capabilities in chemistry and biology were assessed by an external group who conducted red teaming designed to measure the potential scientific and operational risks of the models.
    >
    > A red team composed of different subject matter experts (e.g. biology, chemistry, logistics) were tasked to role play as malign actors who want to conduct a well-defined mission in a scenario that is presented to them resembling an existing prevailing threat environment. Together, these experts probe the model to obtain the most useful information to construct a plan that is feasible within the resource and timing limits described in the scenario. The plan is then graded for both scientific and logistical feasibility. Based on this assessment, GDM addresses any areas that warrant further investigation.
    >
    > External researchers found that the model outputs detailed information in some scenarios, often providing accurate information around experimentation and problem solving. However, researchers found steps were too broad and high level to enable a malicious actor.

This was the "test" for CBRN assistance capabilities.

Note, I don't fault the model team at all for this. I think that present AI-research happened in a very particular environment, and that environment is far removed from the more mundane reality of how threats play out in most parts of the world. In a way, arguably, it's group-think inducing a community-wide failure of imagination and a systemic misunderstanding of reality.

Basically, they are trying to do their best, but they're in over their heads.

The goal of an ethics team will be to provide cover for problems created by decisions or suggestions of an llm. They will need to be able to say "We trained it to consider ethical considerations by " <including something like all the reddit and usenet postings ever>. We tested by <some other inadequate strategy that sounds good to lawyers>.

They will use "industry standard practices", they will have tested it somehow, and will be sure to be on the board of some group that is building toothless "ai ethical tests" for llms.

This reminds me of the marketers who hated programmers and coding in general in the early 2000s era, and fundamentally because the developers were seen as more important in a company. So, you know what they did? They didn't learn programming, they still hated it, instead, they renamed their titles to "hackers". Growth "hacking" would go on to become a term and had nothing to do with hacking - ethical or otherwise. It was a glorified marketing term for marketers and by marketers to justify their exorbitant salaries but also wanted the aura of a real hacker without working for it. I would imagine someone with that title or mindset joining an ethics role in an AI organization is going to find out quickly it isn't all about fluff pieces and grandiose statements at all (we are here to change the world!)

> I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses.

Which like any company will be entirely driven by legal constraints and money. Or just money if it's cheaper to break the law for profit and pay fines. There will be no "this is what's good for humanity, economics be damned".

Much like HR isn't to help employees but just the company.

> The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.

Your disdain is showing.

That you jump immediately to criticizing AI ethicists as counting angels on pins and implying that's why they left OpenAI rather than OpenAI not being a place where AI ethics is a priority is, well, let's say generous to these AI labs.

> My sense is that it is radically shifting from a fluffy marketing arm

Is this your opinion, or do you have something to back it up?

My comment is anecdotal, but everyone I know who works in AI Ethics are not marketing people.

They are data scientists who have moved into the role, as ethics in data has been around long before GenAI. They have to be familiar with the laws around AI and how its going to impact their products. One of them is closer to sort of "HR for AI", in that the focus is not getting the company sued for ethical breaches.

If any of them left, it would be because the company in question is ignoring their direction and they don't want to be there when the shit hits the fan. Plus very few skilled people want to be in a role where they have no control (even if the pay is good).

If OpenAI are hiring marketing people for the role (which I strongly doubt) that would be more troubling IMHO.

Imo 'AI ethics' in this context was code for 'how to make our platform advertiser friendly'

I think you’re confusing Voltaire with the Marquis de Sade and Machiavelli

Read “Careless People” or whatever that book by that Facebook employee that pretended not to see how awful all her colleagues were until her priorities shifted is called.

What does AI ethics even mean?

They can't fully control the model, it does stuff even with instructions not to.

And sometimes it's clear safety mechanisms overcorrect and make the model useless in some situations. I tried asking Claude about some scenario's for a security hole we fixed to see how it would respond, it refused to talk to me seemingly assuming I was trying to introduce a hole that was now fixed. It just wouldn't talk ...

I'm not sure it's a job that you can win at even if you tried / were given all the resources.

AI was trained on human things, including bad things. https://youtu.be/KUXb7do9C-w

There's no safety to be found.

It's not technically possible to prove model alignment. At best you can establish a probability of alignment within certain constraints.

How many of them do you think are actually there to ensure ethical and benevolent AI? They are like the Human Resources department for AI models. They don’t give a shit. Their job is to provide cover for the company and to make some psychopath’s shit circus look like a professional operation.

I regret to say such a book may end up as a pamphlet. I don't see much going into it.

I think AI ethicists will be pushed out entirely after (before?) the money really starts rolling in. Ethical concerns interfere with profit margins.

> My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development

Did you feel this way about social media company liability?

They put on a decent show as well!

It is a company, it doesn't have ethics to align with.

The whole thing is marketing, they have to do some level of morality theatre to placate the pearl-clutchers.

Hahaha, what an insane take. No, anyone that gets in the way of the money machine is going to have to rapidly get out of the way, or get fired.

They all get Timnit Gebru'd sooner or later .. its a fake role tbh

I hate this attitude that ethics and philosophy are "bullshit jobs". At the same time Americans hate any government regulation and are in the process of tearing government's regulatory capacity apart. So all we're left with is the greediest people in the world throwing a nuclear bomb into the machinery of society and everybody else throwing up their hands hoping we're not headed to a The Terminator type future.

[dead]