Right, the article makes it seem that the attackers have the advantage in that they can try to attack everything at once, but the defenders have the advantage in that they each need to defend only one product.

But it is true that if attackers more easily find holes in software that's hard to patch (embedded), that's a problem.

Yes I agree. After a period of turmoil, there'll be a new equilibrium between attackers and defenders.

Considering the new 'meta' that LLMs encode knowledge about existing software but not new ones, I would expect a side effect that newly written software will be inherently less exploitable by LLMs, even if from an actual security perspective, they're worse in design.

You aren't wrong for the most part, but this whole thing of "find me vulns" is not really accurately describing the space.

Finding vulns has almost become sort of like a vibe thing even before LLMs. There would be some security patch that everyone says is critical because it fixes a vulnerability, but the vulnerability is like "under certain conditions, and given physical access to the device, an attacker can craft a special input that crashes the service"... and thats it.

Even stuff like Spectre and Meltdown, which I highly doubt an LLM can find on its own without specifically knowing about speculative execution attacks, are incredibly hard to use. People made a big deal of those being able to be used from javascript, but to actually leak anything of importance you need to know memory layouts, a bunch of other info and so on.

So while an LLM can patch all the up front vulnerabilities, most if not all of those are completely useless to an attacker. Modern systems are incredibly secure.

On the flip side, the stuff that LLM doesn't know about, that can be exploited. For example, assume that log4shell hasn't been found yet, and that log statements by default can pull jni objects from the internet and execute them. The llms would happily write you code with log statements using log4j, and pass it through vulnerability checker, and I would bet that even at the bytecode level it won't ever figure out that vulnerability exists.

And overall, because of Rice theorem, you can't tell if the program is fully exploitable or not without actually running it in some form and way. LLMS can help you with this (but not fully of course) by actually running it and fuzzing inputs and observing memory traces, but even this gets very hard when you introduce things like threading and timed executions which can all affect the result.

And also, the LLMs themselves are an exploit vector now. If you manage to intercept the API calls somehow and insert code or other instruction, you can have the developer essentially put the exploit for you into the code.

So I would say the field is about even.

there are tools already like trufflehog that will find leaked secrets

macos still ships with live github private keys in the library folder

oops i put my god level aws key in js on my website

oops i put the supabase key in frontend

oh no! i am the maintainer of a hosted password manager and i took home the cmek key

The higher the productivity multiplier towards exploiting software, the more developers would find themselves severely outmatched: exploiting software is someone’s full-time job, whereas the engineers already have one—building it.

To express this in numerical terms, let’s consider developer’s incentive to spend effort learning to find and actually finding vulnerabilities in their software (as oppposed to building it) as D, and attacker’s incentive to spend effort exploiting that software as A.

I would say initially A = D × 5 is fair. On one hand, the developer knows their code better. However, their code is open, and most software engineers by definition prefer building (otherwise they would have been pentesters) so that’s where most of their time is going. This is not news, of course, and has been so since forever. The newer factor is attackers working for nation-states, being protected by them, and potentially having figurative guns to their heads or at least livelihoods depending on the amount of damage they can deal; the lack of equivalent pressure on the developer’s side leads me to adjust it to A = D × 10.

×10 is our initial power differential between the attacker and the developer.

Now, let’s multiply that effort by a constant L, reflecting the productivity boost from LLMs. Let’s make it a 10 (I’m sure many would say LLMs make them more tham ×10 more productive in exploit-finding, but let’s be conservative).

Additionally, let’s multiply that by a variable DS/AS that reflects developer’s/attacker’s skill at using LLMs in such particular ways that find the most serious vulnerabilities. As a random guess, let’s say AS = DS × 5, as the attacker would have been exclusively using LLMs for this purpose.

With these numbers substituted in, X would be our new power differential:

    X = (A × L × AS) ÷ (D × L × DS)

    X = (D × 10 × 10 × DS × 5) ÷ (D × 10 × DS)

    X = 50. 

I didn’t account for the fact that many (especially smaller) developers may not even have the resources to run the equivalent compute power as a dedicated hacking team.

Some ways to shift the balance back could be ditching the OSS model and going all-in on the so-called “trusted computing”. Both measures would increase the amount of effort (compute) the attacker may need to spend, but both happen to be highly unpopular as they put more and more power and control in the hand of the corporations that build our computers. In this way, the rise of LLMs certainly advances their interests.

The world runs on legacy software, which simply has no budget for even asking LLMs to find vuls, not to mention fixing them. While you are right that it should favor the defenders, in practice, this is a disaster because, in many critical cases, there are no defenders.