In my personal experience Sol with cyber verification is extremely capable of finding vulnerabilities, and it works even with binaries if you have some kind of IDA/Ghidra CLI access. Of course, unless the binary is protected with Denuvo/VMProtect/etc.
It sounds absurd, but in the last few weeks I've had a few cases where Sol found an RCE in self-hosted web applications in literal minutes just from reading the code (I prefer when it tries to reason statically instead of spamming runtime probes at first).
In another case it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary - any other player in a match could just send you files to anywhere on your system.
I do these things for pure entertainment and curiosity, not for money from bug bounties, so if Sol can find those with a trivial prompt in tens of minutes for me, then what can focused companies/actors find in days or weeks?
Although I think most vulnerabilities are going to be closed in popular software by mid 2027, except in niche old or abandoned projects.
> it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary
Video games are now ruined for me. I don't think I will ever feel safe playing online again.
> I do these things for pure entertainment and curiosity, not for money from bug bounties
Me too... Was it easy to get TAC access? My account isn't even launching the Persona verification, says I'm not eligible.
> Although I think most vulnerabilities are going to be closed in popular software by mid 2027, ...
If we're to believe how good those AI are at CTF and at escapes of all kind, then the only logical conclusion is that, by very far, most vulnerabilities were already closed, even before AI.
Otherwise we'd already be in deep shit since months if not years.
Is cyber verification a thing they're actually doing now? I thought they only reached out to really incredibly famous people and that there's no way to get access as a normal person.
Link to the disclosures then.. prove it. Anyone can say this.. i found an RCE in netBSD using gemma e2b
I was replacing the factory OS from a NAS and asked Sol to do an inventory on the running system over SSH, so that I could replicate any hardware compatibility quirks and whatnot.
It reverse engineered a binary daemon that set fan curves and told me how I should set them up in the new OS. I didn’t ask for this, and I didn’t have reverse engineering tools installed. It just figured out it could run them using Nix.
The most worrying part, to me, is that it did it like it was nothing. It simply said “usr/local/bin/some-daemon sets the following fan curves”. I had to ask how it reached that conclusion for it to tell me casually that it had just read it straight from the x86_64 assembly.
No access to the source code is no longer a meaningful obstacle to these models.