More like AI is much faster at finding obscure vulnerabilities, because it doesn’t get tired of digging into the mind-numbing details of human-generated spaghetti code. Give a human the same kind of task, and they would die of boredom 12 hours into it.
We’re entering an age where AI is quickly cleaning up the stupid security holes that have been created by decades of human mistakes, and will continue to be this way until increased AI security reviews will become more standard. The tribals wielding spears will continue to get shot by superior firepower until they adapt to the same weapons and protection themselves.
“Finding” is also rather generous, going by numbers from Anthropic and from the Linux kernel, things like Mythos are wrong 95% of the time.
They’re much easier to run though, compared to more reliable tools. So even being wrong 95% of the time wins out if you’ve got 100x more people running them.
A 95% false positive rate isn’t actually that big a deal if you have a good way to test the claims. I wonder if you could set it up CtF-style, where you don’t get notified until the LLM can write to a specific file on the target box.
In my experience, the LLM usually proposes a solution, but it’s a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.
I use this shit on a daily basis at work. They work well, as long as you’re choosing the right models, making the right engineering decisions, and have a good set of baseline rules.
Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it’s definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.
“Baseline” is subjective, so I’d recommend just start a global rules file and steer it away from the behaviors you don’t like. Don’t be afraid of quizzing the agent itself on why it did that thing, and what’s a good bit of language to add to the rules file to prevent it from happening.
Also, for specific tasks, skill files are really useful, and LLMs can write the whole thing, based on what you did the hard way in a session. Same thing for triggering a skill file and then ask it if there are any useful improvements to make to the skills file in the future, at the end of the session.
Yeah and this isn’t a new paradigm, in the sense that tools to find exploits have gotten better repeatedly and allowed those with enough knowledge to use the tools to find them, regardless of if they could manually craft the exploit themselves. There’s a multitude of bug bounties that have been claimed by those who knew how to let a reverse engineering tool like ghidra hammer away at something with variation for long enough. And the models are capable of novel exploits on their own, but generally they are just doing the same, hammering ghidra with an MCP at accelerated scale. So “only ai would have found this bug” is technically untrue and moot.
This example demonstrates my point though. It requires an extremely unusual system configuration that may or may not even exist in the real world it is so unusual. AI doesnt get tired, like you said, but its not about spaghetti code (both musl and openssh are gold-standard opposite of spaghetti code), its about trying every possible software combination in ways a human wouldnt bother.
It requires an extremely unusual system configuration that may or may not even exist in the real world it is so unusual.
You have to remember that the same AI agents finding the code are also able to exploit the code. So, if it’s some extremely rare race condition requiring some convoluted pattern, it’s still easy to exploit.
When I stack dominoes in a line on the floor and I push one of them on the outside as it is in the shape of a snake… they fall down in a prdictable pattern. I never knew that would happen until I was shown the first time. This knowlege was past down via print made by humans scraped by bots. For what and for who? Louder faster and dumber… on a foundation of jUsT gEt iT d0Ne… now you are fired. Loops, poops and dupes.
More like AI is much faster at finding obscure vulnerabilities, because it doesn’t get tired of digging into the mind-numbing details of human-generated spaghetti code. Give a human the same kind of task, and they would die of boredom 12 hours into it.
We’re entering an age where AI is quickly cleaning up the stupid security holes that have been created by decades of human mistakes, and will continue to be this way until increased AI security reviews will become more standard. The tribals wielding spears will continue to get shot by superior firepower until they adapt to the same weapons and protection themselves.
“Cleaning up” seems a little generous. It’s finding them, but someone still has to patch them.
“Finding” is also rather generous, going by numbers from Anthropic and from the Linux kernel, things like Mythos are wrong 95% of the time.
They’re much easier to run though, compared to more reliable tools. So even being wrong 95% of the time wins out if you’ve got 100x more people running them.
A 95% false positive rate isn’t actually that big a deal if you have a good way to test the claims. I wonder if you could set it up CtF-style, where you don’t get notified until the LLM can write to a specific file on the target box.
Most vulnerability patches are a few lines of code. Finding it is the hard part.
Even then, the AI session that found it probably already has a solution to fix it.
In my experience, the LLM usually proposes a solution, but it’s a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.
I use this shit on a daily basis at work. They work well, as long as you’re choosing the right models, making the right engineering decisions, and have a good set of baseline rules.
Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it’s definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.
“Baseline” is subjective, so I’d recommend just start a global rules file and steer it away from the behaviors you don’t like. Don’t be afraid of quizzing the agent itself on why it did that thing, and what’s a good bit of language to add to the rules file to prevent it from happening.
Also, for specific tasks, skill files are really useful, and LLMs can write the whole thing, based on what you did the hard way in a session. Same thing for triggering a skill file and then ask it if there are any useful improvements to make to the skills file in the future, at the end of the session.
Yeah and this isn’t a new paradigm, in the sense that tools to find exploits have gotten better repeatedly and allowed those with enough knowledge to use the tools to find them, regardless of if they could manually craft the exploit themselves. There’s a multitude of bug bounties that have been claimed by those who knew how to let a reverse engineering tool like ghidra hammer away at something with variation for long enough. And the models are capable of novel exploits on their own, but generally they are just doing the same, hammering ghidra with an MCP at accelerated scale. So “only ai would have found this bug” is technically untrue and moot.
This example demonstrates my point though. It requires an extremely unusual system configuration that may or may not even exist in the real world it is so unusual. AI doesnt get tired, like you said, but its not about spaghetti code (both musl and openssh are gold-standard opposite of spaghetti code), its about trying every possible software combination in ways a human wouldnt bother.
You have to remember that the same AI agents finding the code are also able to exploit the code. So, if it’s some extremely rare race condition requiring some convoluted pattern, it’s still easy to exploit.
When I stack dominoes in a line on the floor and I push one of them on the outside as it is in the shape of a snake… they fall down in a prdictable pattern. I never knew that would happen until I was shown the first time. This knowlege was past down via print made by humans scraped by bots. For what and for who? Louder faster and dumber… on a foundation of jUsT gEt iT d0Ne… now you are fired. Loops, poops and dupes.