logoalt Hacker News

Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery

234 pointsby malsheyesterday at 7:47 PM191 commentsview on HN

Comments

craftkilleryesterday at 9:20 PM

I've noticed on YouTube I see a lot of adult sexual ads. Things like home cleaning ads that have an unmistakable wood-grain image of fellatio. I've always wondered "how are these getting past the moderators?". I guess the answer is the obvious one: no one is moderating anything.

show 10 replies
CrzyLngPwdyesterday at 8:16 PM

All their fines are merely a cost of doing business, and they won't change their ways until the fines hurt.

show 2 replies
xhevahirtoday at 12:23 AM

If stuff like this shows up in GAI I wonder what that means for how common such material is in the wild. Frightening.

thi2today at 12:07 AM

Metas ads are all over the place. Thomas Kehl, one of the founders of "Finanzfluss" (a german site/content creation company about finance) is currently suing meta because scammers keep using his picture.

There really needs to be some harder punishments for this kind of behavior, after all meta profits by selling scammy ads.

callamdelaneyyesterday at 10:43 PM

Meta is still running ads that suggest violence against politicians, when reported they say it does not contravene their policies.

show 1 reply
Kalanosyesterday at 9:03 PM

What's crazy is how long it takes to report something like this to a big company and actually have them take action on it. I remember 10 years ago I reported a Giphy ad for the same thing, and it took them months to take action.

show 4 replies
onlyrealcuzzoyesterday at 10:03 PM

Guys, it's fine.

Relax.

It's only a crime if someone else does it or is not sufficiently wealthy, otherwise it's a mistake.

froobiusyesterday at 9:14 PM

I'm starting to wonder if any of this is better than local newspapers, where there's an editor with oversight over all content...

type0yesterday at 8:49 PM

So that's why they are lobbying for age verification, i.e ID verification?

show 2 replies
godwinson__4-8yesterday at 9:08 PM

Somewhat related: open YouTube in an incognito window.

Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad.

The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage.

When I compare to my normal/auth'd Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.

show 5 replies
yaloginyesterday at 10:11 PM

Even if they are caught and it’s illegal, as long as the fine for it is less than the roi of running them the companies will not stop.

meric_yesterday at 9:18 PM

I'm wondering what people expect these companies to do? Human moderators? Sure, but Meta products have approximately half the worlds population using them. At this scale it's pretty mind bogglingly difficult to moderate down everything. The ads/posts served per second is an astronomical number.

This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up

show 28 replies
shitloadofbooksyesterday at 9:44 PM

Meta will do whatever it can to run any and all ads.

Meta knowingly runs scam ads, they just charge more to run them: https://www.reuters.com/legal/litigation/consumer-watchdog-g...

> Reuters also reported that Meta internally projected in 2024 that it would earn about 10% of its overall annual revenue – or $16 billion – from running ads for scams and banned goods, internal company documents show.

At $employer we're CONSTANTLY fighting fake ads cloning our brand. Their various brand protection features seem deliberately inept and ineffective.

lp4v4nyesterday at 8:46 PM

I'd be interested to know if the same amount of leniency shown here would be applied if we were talking about ads depicting violence against elected politicians.

show 2 replies
randyrandyesterday at 10:52 PM

Non-specific terms are a pet peeve of mine in accusations.

Is this new term broader than the old term of CP or is it the same? Why did we even switch?

show 2 replies
dotcomayesterday at 9:01 PM

Classy as usual.

xgulfieyesterday at 9:18 PM

Weird how they never thought to require ID verification for advertisers hmmm

malsheyesterday at 8:25 PM

https://archive.ph/4LhoU

I didn't realize the original was behind a paywall.

disillusionedyesterday at 10:01 PM

Beyond these obviously deplorable ads, my wife got sniped by an ecommerce lookalike ad on Google the other day: a near-identical match of the vendor she was looking for, with a paid for sponsored ad (that was clearly linked to some chinese firm upon further review). Site had prices that were just a bit too good to be true, and a slightly different TLD that _I_ would've caught, but she missed because the price was a bit too good to be true. Only caught it when Citi asked me if I was spending 9000EGD on some random website.

And my point here is, we're in this great revolution of AI empowered everything and... these companies can't be bothered to scan this volume of AI ad slop to make sure we're not being actively scammed? Or having porn or CSAM spread? I understand it's a "perverse incentive" in the truest sense: they make money by taking the scammers money, but I was scammed once by an insanely clever lookalike Facebook ad, and had trusted their verified checkmark (about a decade ago)... with AI, these things should be a thing of the past.

stephenhueyyesterday at 8:04 PM

Non-paywall alternative:

https://www.engadget.com/2231100/meta-apps-displayed-ads-tha...

show 1 reply
crmdyesterday at 11:28 PM

I asked my grandpa about the business idea of self-serve magazine advertising. He said if you do that wait till you see the weird shit people will try to publish.

I wonder if we could learn anything from these cranky old timers.

andy_pppyesterday at 9:38 PM

If any of us did that clearly we’d be in jail for several years and put on the sex offenders register but when Meta appear to be hugely responsible for increases in teen suicide or crimes like these we get nothing. If corporations want the rights of individuals they equally should shoulder our shared societal responsibilities as well. I honestly don’t get how they can just pretend they aren’t responsible somehow because it’s slightly expensive to not commit crimes or something…

If you are downvoting this please explain why, it’s important that we come to a shared understanding how these things are ever okay for corporations and not individuals…

show 2 replies
lenerdenatoryesterday at 9:11 PM

This is what happens when you let a company be controlled by one person who has no reason to be accountable to anyone.

If Zuckerberg wants total voting control, then he can also deal with the consequences of being caught hosting CSAM.

show 1 reply
2OEH8eoCRo0yesterday at 11:42 PM

Repeal Section 230

steveBK123yesterday at 8:07 PM

grok is this you

show 1 reply
snitzryesterday at 8:07 PM

Shut it all down.

bigbuppoyesterday at 9:19 PM

As if you didn't need another reason to use an ad blocker.

show 1 reply
alex1138yesterday at 9:02 PM

> and ran across Facebook, Threads, Messenger and Instagram.

So it's the SAME AD PLATFORM? Even in MESSENGER? My god. Not only is this whole thing horrible but it's doubly so

chrisjjyesterday at 9:32 PM

> AI-generated

How on earth can they know?

show 1 reply
ToucanLoucanyesterday at 8:08 PM

> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through.

Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop.

Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.

show 7 replies
josefritzishereyesterday at 8:24 PM

Zuckerberg needs to be arrested and imprisoned. These sociopaths aren't going to understand right and wrong on their own. It requires an intervention.

show 1 reply
eboyyesterday at 11:30 PM

[dead]

daun_geeyesterday at 8:23 PM

[flagged]

show 1 reply
alex1138yesterday at 8:15 PM

(This reminds me of, and not to be political. This is a technological, rather than political angle) There's a TED talk by Carole Cadwalladr about "Facebook's role in Brexit" in which she talks about the ads the Leave campaign ran. This is what is meant by Facebook's "breaking democracy", where campaigns don't have to go through the usual disclosure avenues

Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook?

The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape

This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this

cdrnsfyesterday at 10:46 PM

We were better off with landlines and local journalism.

throwatdem12311yesterday at 11:33 PM

Just shut the company down, throw the executives in a deep dark hole and forget they exist. The world will be better off.