My first thought was that this was about hardening critical software against AI drift-induced architectural brittleness. I have a feeling that’s going to become a real problem in the next few...
My first thought was that this was about hardening critical software against AI drift-induced architectural brittleness. I have a feeling that’s going to become a real problem in the next few years, particularly with SaaS platforms, as agentic engineers keep implementing new features as afterthoughts tacked onto the existing codebase, rather than properly integrating them into those systems in a cohesive way.
I think this is going to be a major issue that companies will be unwilling to acknowledge or talk about. Somebody will probably invent a word for it. From the outside, it’s just going to look like our software is just getting slower, maybe weird regressions start appearing, maybe UIs start introducing bizarre deviations from their familiar UX patterns and branding. It’s mostly going to be only half-perceived like “am I going crazy or did this feature work better last month?” People who point out the drift will be gaslit and told that’s conspiracy thinking. Because nobody outside those companies has eyes on the code and everybody imagines the companies to be competent and, well, capable of controlling the development of their flagship products.
But they won’t be. Inch by inch the AI’s myopic decisions are going to paint these codebases into corners. Code will slowly spaghettify, tech debt will spread like cancer. There won’t be a way to reverse course without major rewrites. Engineering teams will just compound the problem by directing the AI to play whac-a-mole with bug reports, tacking on more band-aid fixes instead of untangling the Gordian knot.
That’s not the problem Project Glasswing is trying to solve. This is about using AI to proactively, defensively find security holes and patch them before they can be exploited. Which is also hugely important, and I’m very much in favor of the initiative.
There’s a lot of enterprise legacy code out there that was written without AI, so that situation doesn’t seem new. It’s not easy to refactor your way out, but it’s possible, and coding agents...
There’s a lot of enterprise legacy code out there that was written without AI, so that situation doesn’t seem new. It’s not easy to refactor your way out, but it’s possible, and coding agents might help?
I think as long as you spend some time asking the AI to clean up the code, you’re less likely to get stuck like that.
Right? This isn’t 2023 anymore. The industry has established patterns to be able to use AI correctly. So that code doesn’t just become mush. Can it still? For sure. And the same code bases that...
Right? This isn’t 2023 anymore. The industry has established patterns to be able to use AI correctly. So that code doesn’t just become mush. Can it still? For sure. And the same code bases that would have otherwise been indecipherable and unscablable will still be as such. But those who look for best practices can use new best practices.
As a member of that industry (albeit between jobs at the moment) who is pretty deep in the agentic engineering space these days, I'd like to know more about those best practices you're talking...
As a member of that industry (albeit between jobs at the moment) who is pretty deep in the agentic engineering space these days, I'd like to know more about those best practices you're talking about. From where I'm sitting that still seems very much up in the air.
Yes, it seems more like gossip than conventional wisdom at that point. A lot of people are building their own infrastructure and coming up with their own best practices based on what seems to work...
Yes, it seems more like gossip than conventional wisdom at that point. A lot of people are building their own infrastructure and coming up with their own best practices based on what seems to work and by trying things they've heard about.
But I expect that the industry will learn things and they will be written down, codified as open source software and become more widely known. It seems a little odd to bet against people learning how to use tools better.
It might take a while, like what happened with JavaScript frameworks.
Okay, I see where you're coming from now. I'm not making a long-term bet against it... what I'm predicting is a near-term crisis that acts as an industry wake-up call, which in turn leads to much...
Okay, I see where you're coming from now. I'm not making a long-term bet against it... what I'm predicting is a near-term crisis that acts as an industry wake-up call, which in turn leads to much better (I probably mean "deterministic") tools and workflows that really do solve the problem for good. But right now? It's still the wild west. In the meantime I think a lot of engineers, or engineering departments, are going to get overconfident and create some real messes for themselves, before actual best practices are solidified.
From what I've seen, not that I've seen a ton of other people's use of AI first-hand, some engineers enter a state where they think much less deeply simply because an LLM is typing code for them....
From what I've seen, not that I've seen a ton of other people's use of AI first-hand, some engineers enter a state where they think much less deeply simply because an LLM is typing code for them. If you are still engineering your software like an adult and think through migrations needed and then walk the LLM through that, or properly redesign the UI and business logic when requirements change, then you are getting all of the upsides of LLM-assisted programming without much of the downsides. If you just bark at it to add features, poke at the UI and complain back only when you can tell something is wrong, then that's a recipe for spaghetti code. For products people are going to depend on "I don't even look at the code anymore" is not going to cut it.
Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser. Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe. Project Glasswing is an urgent attempt to put these capabilities to work for defensive purposes.
As part of Project Glasswing, the launch partners listed above will use Mythos Preview as part of their defensive security work; Anthropic will share what we learn so the whole industry can benefit. We have also extended access to a group of over 40 additional organizations that build or maintain critical software infrastructure so they can use the model to scan and secure both first-party and open-source systems. Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations.
...
Over the past few weeks, we have used Claude Mythos Preview to identify thousands of zero-day vulnerabilities (that is, flaws that were previously unknown to the software’s developers), many of them critical, in every major operating system and every major web browser, along with a range of other important pieces of software.
...
We do not plan to make Claude Mythos Preview generally available, but our eventual goal is to enable our users to safely deploy Mythos-class models at scale—for cybersecurity purposes, but also for the myriad other benefits that such highly capable models will bring. To do so, we need to make progress in developing cybersecurity (and other) safeguards that detect and block the model’s most dangerous outputs. We plan to launch new safeguards with an upcoming Claude Opus model, allowing us to improve and refine them with a model that does not pose the same level of risk as Mythos Preview.
The more concerning part is the competitive edge that these 40 organizations are going to get over others in their industries. Nonetheless that this could almost cause inadvertent coalition...
The more concerning part is the competitive edge that these 40 organizations are going to get over others in their industries. Nonetheless that this could almost cause inadvertent coalition building if Mythos all directs them in one direction? Weird things
I expect they'll open it up more after major browsers and OSes are hardened. Also, people at these companies can contribute patches to open source projects.
I expect they'll open it up more after major browsers and OSes are hardened. Also, people at these companies can contribute patches to open source projects.
Earlier I heard this sentiment in another commentary of the announcement as well and immediately wondered: Are major browsers and OS’s ever going to be “done” hardening? Like, when do Anthropic or...
after major browsers and OSes are hardened
Earlier I heard this sentiment in another commentary of the announcement as well and immediately wondered: Are major browsers and OS’s ever going to be “done” hardening?
Like, when do Anthropic or the companies in the Glasswing consortium decide that now they’ve done it, no more zero-days in Chrome, Linux, iOS, Firefox, *BSD and so on? You can’t even approximate a threshold of “acceptable percentage of memory exploits (etc.) fixed/prevented” that, once crossed, is sufficient to release Mythos to the general public, since the number at the ceiling is completely in the realm of “unknown unknowns”.
The only conclusion I see at the moment is that this model never sees the light of day outside of very purposeful use behind closed doors, but we know that’s (at least very very likely) not going to be the case from a business perspective. Or at least not for a couple of years, until they run out of exploits to patch – if it’s really as powerful in exploring new bugs and faults as the announcement leads us to believe.
You could aim for provably correct code I suppose. It's a very high bar and used to not be out of reach for all but very simple or highly critical software. I can see agentic coding doing that...
You could aim for provably correct code I suppose. It's a very high bar and used to not be out of reach for all but very simple or highly critical software. I can see agentic coding doing that sort of thing more cheaply, but it is still probably not worth it for most software tbh.
The rate at which new security bugs are found by Mythos seems measurable. It seems like it would slow down a lot after they fix the ones it finds easily. After it hasn't found anything new for a...
The rate at which new security bugs are found by Mythos seems measurable. It seems like it would slow down a lot after they fix the ones it finds easily. After it hasn't found anything new for a while, it would be safer to release it. (At least for major OSes and browsers.)
The tricky bit: what if Mythos gets better when someone improves the harness that runs it? The LLM is an important part of of the system, but the other parts matter too.
The worst case is something like "eternal September" where reports of new security bugs never slows down. I imagine that would only happen for a project where the software development process is somehow cursed, not for code that's reasonably well-engineered.
My first thought was that this was about hardening critical software against AI drift-induced architectural brittleness. I have a feeling that’s going to become a real problem in the next few years, particularly with SaaS platforms, as agentic engineers keep implementing new features as afterthoughts tacked onto the existing codebase, rather than properly integrating them into those systems in a cohesive way.
I think this is going to be a major issue that companies will be unwilling to acknowledge or talk about. Somebody will probably invent a word for it. From the outside, it’s just going to look like our software is just getting slower, maybe weird regressions start appearing, maybe UIs start introducing bizarre deviations from their familiar UX patterns and branding. It’s mostly going to be only half-perceived like “am I going crazy or did this feature work better last month?” People who point out the drift will be gaslit and told that’s conspiracy thinking. Because nobody outside those companies has eyes on the code and everybody imagines the companies to be competent and, well, capable of controlling the development of their flagship products.
But they won’t be. Inch by inch the AI’s myopic decisions are going to paint these codebases into corners. Code will slowly spaghettify, tech debt will spread like cancer. There won’t be a way to reverse course without major rewrites. Engineering teams will just compound the problem by directing the AI to play whac-a-mole with bug reports, tacking on more band-aid fixes instead of untangling the Gordian knot.
That’s not the problem Project Glasswing is trying to solve. This is about using AI to proactively, defensively find security holes and patch them before they can be exploited. Which is also hugely important, and I’m very much in favor of the initiative.
But the other thing needs to be solved too.
There’s a lot of enterprise legacy code out there that was written without AI, so that situation doesn’t seem new. It’s not easy to refactor your way out, but it’s possible, and coding agents might help?
I think as long as you spend some time asking the AI to clean up the code, you’re less likely to get stuck like that.
Right? This isn’t 2023 anymore. The industry has established patterns to be able to use AI correctly. So that code doesn’t just become mush. Can it still? For sure. And the same code bases that would have otherwise been indecipherable and unscablable will still be as such. But those who look for best practices can use new best practices.
As a member of that industry (albeit between jobs at the moment) who is pretty deep in the agentic engineering space these days, I'd like to know more about those best practices you're talking about. From where I'm sitting that still seems very much up in the air.
“Where are these best practices they speak of? Are they in the room with us now?”
Sorry, I know it’s a noisy comment, but I just had to.
Yes, it seems more like gossip than conventional wisdom at that point. A lot of people are building their own infrastructure and coming up with their own best practices based on what seems to work and by trying things they've heard about.
But I expect that the industry will learn things and they will be written down, codified as open source software and become more widely known. It seems a little odd to bet against people learning how to use tools better.
It might take a while, like what happened with JavaScript frameworks.
Okay, I see where you're coming from now. I'm not making a long-term bet against it... what I'm predicting is a near-term crisis that acts as an industry wake-up call, which in turn leads to much better (I probably mean "deterministic") tools and workflows that really do solve the problem for good. But right now? It's still the wild west. In the meantime I think a lot of engineers, or engineering departments, are going to get overconfident and create some real messes for themselves, before actual best practices are solidified.
From what I've seen, not that I've seen a ton of other people's use of AI first-hand, some engineers enter a state where they think much less deeply simply because an LLM is typing code for them. If you are still engineering your software like an adult and think through migrations needed and then walk the LLM through that, or properly redesign the UI and business logic when requirements change, then you are getting all of the upsides of LLM-assisted programming without much of the downsides. If you just bark at it to add features, poke at the UI and complain back only when you can tell something is wrong, then that's a recipe for spaghetti code. For products people are going to depend on "I don't even look at the code anymore" is not going to cut it.
From the article:
...
...
The more concerning part is the competitive edge that these 40 organizations are going to get over others in their industries. Nonetheless that this could almost cause inadvertent coalition building if Mythos all directs them in one direction? Weird things
I expect they'll open it up more after major browsers and OSes are hardened. Also, people at these companies can contribute patches to open source projects.
Earlier I heard this sentiment in another commentary of the announcement as well and immediately wondered: Are major browsers and OS’s ever going to be “done” hardening?
Like, when do Anthropic or the companies in the Glasswing consortium decide that now they’ve done it, no more zero-days in Chrome, Linux, iOS, Firefox, *BSD and so on? You can’t even approximate a threshold of “acceptable percentage of memory exploits (etc.) fixed/prevented” that, once crossed, is sufficient to release Mythos to the general public, since the number at the ceiling is completely in the realm of “unknown unknowns”.
The only conclusion I see at the moment is that this model never sees the light of day outside of very purposeful use behind closed doors, but we know that’s (at least very very likely) not going to be the case from a business perspective. Or at least not for a couple of years, until they run out of exploits to patch – if it’s really as powerful in exploring new bugs and faults as the announcement leads us to believe.
You could aim for provably correct code I suppose. It's a very high bar and used to not be out of reach for all but very simple or highly critical software. I can see agentic coding doing that sort of thing more cheaply, but it is still probably not worth it for most software tbh.
The rate at which new security bugs are found by Mythos seems measurable. It seems like it would slow down a lot after they fix the ones it finds easily. After it hasn't found anything new for a while, it would be safer to release it. (At least for major OSes and browsers.)
The tricky bit: what if Mythos gets better when someone improves the harness that runs it? The LLM is an important part of of the system, but the other parts matter too.
The worst case is something like "eternal September" where reports of new security bugs never slows down. I imagine that would only happen for a project where the software development process is somehow cursed, not for code that's reasonably well-engineered.
Major software companies are all involved in multiple trade associations, particularly in the security space. This doesn’t seem unusual to me?