logoalt Hacker News

baqyesterday at 6:58 AM8 repliesview on HN

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)

As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?


Replies

visargayesterday at 7:16 AM

Right now I tried "What is digestion?" -> "Fable 5's safeguards flagged this message. Our intentionally broad safeguards deliver more capabilities but can also flag safe coding, cybersecurity, and biology tasks. Send feedback or learn more."

I don't think you can 100% ensure your queries have absolutely no biology and cyber keywords inside. No matter how harmless, they always trigger. People complain Fable aborts even when they try to make a login page for showing "username" and "password".

This makes models like Fable 5 impossible to use in any serious agentic task, because you can't even guarantee the model, which is a basic thing you need to build on.

show 3 replies
compass_copiumyesterday at 10:58 AM

Not to harp on you (already being downvoted to oblivion for expressing a reasonable and common opinion), but the whole conversation about LLMs enabling bioterrorism or explosive manufacturing is a bit silly. The hard part of making anthrax or sarin or whatever isn't finding a recipe, it's getting (scheduled, controlled) precursors, (monitored, traced) equipment and manufacturing skills. The information is there. It's already easy to get, it's the physical materials that are more difficult.

Also, if you live in America, it is much easier and more effective to create a mass casualty event with, say, a few cases of fireworks and a pressure cooker or an AR-15.

show 3 replies
egorfineyesterday at 10:13 AM

> I don’t want anyone to have the capability

I don't want anyone to have the capability to rape women.

show 2 replies
ethinyesterday at 3:34 PM

This is nonsense. By this logic some random corporation should have total control over your computer and the inputs you feed it and the outputs it produces to ensure nobody who isn't "verified crazy" uses it. That's essentially what your saying.

These models are, ultimately, tools. I would never trust some random corporation (particularly one with a profit motive and hypocritical stance, which includes both OpenAI and Anthropic, to be clear) to decide what isn't and is considered "crazy" and who and who isn't "verified" not to be "crazy". Especially when these companies have time and time again demonstrated (1) that they cry wolf way too much which leads to nobody taking their claims about how "dangerous" their models are seriously and (2) incidents like this where OpenAI makes a claim ("Look at how dangerous our models are!") and then doesn't be smart and just... Slow the fuck down (and when testing these things, actually sandbox them properly, which obviously wasn't done here or this attack wouldn't have been even possible).

jrs100000yesterday at 7:13 AM

Llama isn't going to invent shit. It wont be able to tell you anything accurate that you couldn't get out of a chemistry textbook.

show 1 reply
benj111yesterday at 10:28 AM

Anyone who has the skills to create a novel bioweapon has the skills to recreate lots of ones we have already.

Any physics teacher should know the theory for constructing a nuclear bomb. Should we be controlling that knowledge too?

What about flight simulators? Don't want a load of people knowing how to fly.

This isn't computer science, the hard bit is getting the materials and equipment, not the knowledge.

show 2 replies
foxglacieryesterday at 8:04 AM

Never heard of hemlock or mushrooms? Crazy people already have! Run for the hills!