3 months ago I watched this guy announce the end of the bug bounty program due to slop on stage. Have the tools gotten that much better, or is it just that without the profit motive, people are...
3 months ago I watched this guy announce the end of the bug bounty program due to slop on stage. Have the tools gotten that much better, or is it just that without the profit motive, people are spending more time separating the real vulns from the slop?
I don't know, but I would guess a little of both. Finding a security vulnerability in curl would be a big deal for anyone professionally. So unfortunately low effort scan that clogs up the...
I don't know, but I would guess a little of both. Finding a security vulnerability in curl would be a big deal for anyone professionally. So unfortunately low effort scan that clogs up the security teams time was probably the main reason for stopping the bug bounty program. If all the issues posted had been for real vulnerabilities I would hope they'd be scrambling to patch them and kept the program running. In addition Mythos is apparently very capable (we have to trust anthropics word on this since there's no way for us mortals to verify it). Given the capability of the models have improved the quality of the reported issues would likely go up. Question is if it's worth having a bug bounty program that will essentially just be receiving slop (unless you have some automated way of verifying the found issues).
Hazarding a guess, the bug bounty program will remain closed. Quality of the reports for bugs/vulnerabilities will probably go up.
There's been a marked improvement in the output that just about everyone in the industry noticed about a month or so after that announcement. It does seem like some of it was just getting enough...
There's been a marked improvement in the output that just about everyone in the industry noticed about a month or so after that announcement.
It does seem like some of it was just getting enough training data/testing to clean things up and improve models.
If hackers can “insert” code directly into the codebase it’s already joever. That’s like discussing what lock works best for your house in the circumstance that the thief is already in your house.
If hackers can “insert” code directly into the codebase it’s already joever. That’s like discussing what lock works best for your house in the circumstance that the thief is already in your house.
Code goes through review before it's accepted into a project. This goes for anything as soon as more than one dev is involved. "Seemingly innocent code" is still checked and caught. Yes, there are...
Code goes through review before it's accepted into a project. This goes for anything as soon as more than one dev is involved. "Seemingly innocent code" is still checked and caught. Yes, there are a few cases where it has slipped through - but that's a few cases among millions of commits and pull requests.
Open-source means you can access a copy of the source code. It does not mean anybody can freely modify what is distributed.
I think you're overestimating the degree to which an exploit can be engineered to explicitly get around AI code review systems. There is a risk of missing a potential exploit being sneakily...
I think you're overestimating the degree to which an exploit can be engineered to explicitly get around AI code review systems. There is a risk of missing a potential exploit being sneakily inserted into your code in a PR, but that risk exists with humans reviewing and merging PRs too. If you still have humans in the loop, it's not clear to me how also using an AI tool designed to detect hidden vulnerabilities makes your software any more at risk than it would be with solely human oversight. Someone who's trying to play the long game to insert a vulnerability like that is already going to be designing their work to evade human scrutiny, after all.
A couple points. First, only a select few currently have access to this model. There’s absolutely a danger of bad actors using AI to find vulnerabilities to exploit, which is why (for now)...
A couple points.
First, only a select few currently have access to this model. There’s absolutely a danger of bad actors using AI to find vulnerabilities to exploit, which is why (for now) Anthropic’s only letting the known good guys use it.
Second, the scenario you’re describing would require hackers to have contributor access to the code repository. That almost never happens… it would indicate a significant compromise before a single malicious change was authored, and in cases where that sort of thing happens the changes are usually spotted in PR, in human code review. It’s possible to hide or disguise evil code that can sneak through a code review but with AI now assisting in that process I think the odds of pulling it off are vanishingly small. Especially in cornerstones of the FOSS world, like curl is.
Edit: What @DeaconBlue said (and typed faster, lol).
There was this well known attack on xz which kind of highlights how vulnerable some of the worlds software is to a disciplined threat actor. It's not inconceivable that someone is running such...
There was this well known attack on xz which kind of highlights how vulnerable some of the worlds software is to a disciplined threat actor. It's not inconceivable that someone is running such long cons primarily using LLM agents at this point.
To add -- for now at least, Anthropic has restricted use of Mythos to particular parties, most folks shouldn't be able to have access. Anthropic considers its potential for misuse a security...
To add -- for now at least, Anthropic has restricted use of Mythos to particular parties, most folks shouldn't be able to have access. Anthropic considers its potential for misuse a security concern, though the author feels this is more marketing than truth. (The article covers this at the start, pardon if I'm reiterating something you already read)
3 months ago I watched this guy announce the end of the bug bounty program due to slop on stage. Have the tools gotten that much better, or is it just that without the profit motive, people are spending more time separating the real vulns from the slop?
I don't know, but I would guess a little of both. Finding a security vulnerability in curl would be a big deal for anyone professionally. So unfortunately low effort scan that clogs up the security teams time was probably the main reason for stopping the bug bounty program. If all the issues posted had been for real vulnerabilities I would hope they'd be scrambling to patch them and kept the program running. In addition Mythos is apparently very capable (we have to trust anthropics word on this since there's no way for us mortals to verify it). Given the capability of the models have improved the quality of the reported issues would likely go up. Question is if it's worth having a bug bounty program that will essentially just be receiving slop (unless you have some automated way of verifying the found issues).
Hazarding a guess, the bug bounty program will remain closed. Quality of the reports for bugs/vulnerabilities will probably go up.
There's been a marked improvement in the output that just about everyone in the industry noticed about a month or so after that announcement.
It does seem like some of it was just getting enough training data/testing to clean things up and improve models.
Which links to https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/ where he basically addresses your question.
If hackers can “insert” code directly into the codebase it’s already joever. That’s like discussing what lock works best for your house in the circumstance that the thief is already in your house.
Code goes through review before it's accepted into a project. This goes for anything as soon as more than one dev is involved. "Seemingly innocent code" is still checked and caught. Yes, there are a few cases where it has slipped through - but that's a few cases among millions of commits and pull requests.
Open-source means you can access a copy of the source code. It does not mean anybody can freely modify what is distributed.
I think you're overestimating the degree to which an exploit can be engineered to explicitly get around AI code review systems. There is a risk of missing a potential exploit being sneakily inserted into your code in a PR, but that risk exists with humans reviewing and merging PRs too. If you still have humans in the loop, it's not clear to me how also using an AI tool designed to detect hidden vulnerabilities makes your software any more at risk than it would be with solely human oversight. Someone who's trying to play the long game to insert a vulnerability like that is already going to be designing their work to evade human scrutiny, after all.
That would require the maintainers of a project to approve of the code change request. In the case of curl this would be a very high barrier.
A couple points.
First, only a select few currently have access to this model. There’s absolutely a danger of bad actors using AI to find vulnerabilities to exploit, which is why (for now) Anthropic’s only letting the known good guys use it.
Second, the scenario you’re describing would require hackers to have contributor access to the code repository. That almost never happens… it would indicate a significant compromise before a single malicious change was authored, and in cases where that sort of thing happens the changes are usually spotted in PR, in human code review. It’s possible to hide or disguise evil code that can sneak through a code review but with AI now assisting in that process I think the odds of pulling it off are vanishingly small. Especially in cornerstones of the FOSS world, like curl is.
Edit: What @DeaconBlue said (and typed faster, lol).
There was this well known attack on xz which kind of highlights how vulnerable some of the worlds software is to a disciplined threat actor. It's not inconceivable that someone is running such long cons primarily using LLM agents at this point.
To add -- for now at least, Anthropic has restricted use of Mythos to particular parties, most folks shouldn't be able to have access. Anthropic considers its potential for misuse a security concern, though the author feels this is more marketing than truth. (The article covers this at the start, pardon if I'm reiterating something you already read)