skybrian's recent activity
-
Comment on Most US doctors are quietly using the OpenEvidence AI tool. Few patients know about it. in ~health
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentThe White House meeting was apparently to discuss cybersecurity risks for banks, which seems reasonable. Nuclear weapons aren't mentioned on that page. The second paragraph from the system card...The White House meeting was apparently to discuss cybersecurity risks for banks, which seems reasonable. Nuclear weapons aren't mentioned on that page.
The second paragraph from the system card seems like the clearest justification for not releasing Mythos:
In particular, it has demonstrated powerful cybersecurity skills, which can be used for both
defensive purposes (finding and fixing vulnerabilities in software code) and offensive
purposes (designing sophisticated ways to exploit those vulnerabilities). It is largely due to
these capabilities that we have made the decision not to release Claude Mythos Preview for
general availability. -
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentPeople have speculated about such things, but that's not why Anthropic is holding Mythos back.People have speculated about such things, but that's not why Anthropic is holding Mythos back.
-
Comment on Decluttering X and Bsky feeds in ~tech
skybrian LinkI don't have anything to recommend, but Bluesky has an open API so it seems like it should be possible to build an alternate client. It might be a fun project for someone?I don't have anything to recommend, but Bluesky has an open API so it seems like it should be possible to build an alternate client. It might be a fun project for someone?
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentI have no special knowledge about Mythos and I'm not going to make any specific claims about it. Instead, I'm making a burden-of-proof argument: Imagine a car manufacturer decided to delay the...I have no special knowledge about Mythos and I'm not going to make any specific claims about it. Instead, I'm making a burden-of-proof argument:
Imagine a car manufacturer decided to delay the release of a new car due to safety concerns. How well would these arguments hold up?
- "they clearly didn't think the previous model was dangerous"
- "Every other car on the road was safe for release"
- "I test drove it and didn't notice anything particularly dangerous about it"
If a manufacturer claims that their new product has safety issues, usually we take their word for it. We assume they've tested their product better than us and give them the benefit of the doubt. We don't demand extraordinary evidence.
If other people have tested the LLM and they didn't find anything, sure, those are useful observations too and we can infer that the problems aren't immediately obvious. But it doesn't invalidate Anthropic's observations, because there are lots of different ways to use an LLM, such as different prompts, just to start. Maybe they tested something different? It's not enough to prove a lie.
Also, I've been putting it in binary terms, but safety is a continuum. Cars aren't all equally safe, are they? As we're seeing, neither are LLM's. It's looking like it's too late to keep attackers from discovering lots of security bugs using already-released LLM's, so maybe the already-released LLM's weren't safe, for some definitions of "safe?" But I still appreciate that Anthropic makes some effort at testing each new LLM for safety and that they're willing to delay a release if they have concerns.
-
Comment on Multiple security bugs in Dnsmasq in ~comp
skybrian LinkFrom the article:From the article:
There has been something of a revolution in AI-based security research, and I've spent a lot of time over the last couple of months dealing with bug reports, weeding duplicates (so many duplicates!) and triaging bugs into those which need vendor pre-disclosure and those which it's better to make public and fix immediately. Those judgements have been necessarily subjective, but given the number of times "good guys" have found these bugs, there's no doubt that "bad guys" have been able to do the same, so long embargoes seem kind of pointless. There's also the problem that the amount of time and effort, for all actors, needed to co-ordinate an embargo and provide backports is huge. I think the priority for most bugs is to fix them going forward, and have new dnsmasq releases as bug-free as possible. To this end, you may have noticed that there have been a lot of security-fix commits to the git repo in the weeks prior to this announcement.
-
Multiple security bugs in Dnsmasq
9 votes -
Comment on ‘It’s shameful’: New York’s elite lash out at Zohran Mamdani’s second-home tax in ~finance
skybrian Link ParentNot sure what kind of investment you mean, but I imagine investing locally would result in gentrification and higher property values. It doesn't seem like the cure for affordability, unless it's...Not sure what kind of investment you mean, but I imagine investing locally would result in gentrification and higher property values. It doesn't seem like the cure for affordability, unless it's new construction? Investing in existing property (remodelling, etc) makes it more expensive.
-
Comment on Introducing Googlebook, designed for Gemini Intelligence in ~tech
skybrian Link ParentGoogle has built high-end Chromebooks before (like the Pixelbook) and there have been previous kludgy attempts to merge Android and ChromeOS. I imagine this is their latest attempt. Maybe this...Google has built high-end Chromebooks before (like the Pixelbook) and there have been previous kludgy attempts to merge Android and ChromeOS. I imagine this is their latest attempt. Maybe this time it will work?
It will certainly run Chrome. I wonder whether you will be able to turn the AI stuff off?
They are partnering with other manufacturers like they do for Chromebooks, so it seems more like an evolution and rebranding exercise?
-
Comment on The boy that cried Mythos in ~comp
skybrian (edited )Link ParentThat's not a hard fact, it's a judgement call. They think it's dangerous, so they're not releasing it (yet). Obviously, they could, but they chose not to, and it's their decision to make. Other..."This model is so dangerous we can't release it"
That's not a hard fact, it's a judgement call. They think it's dangerous, so they're not releasing it (yet). Obviously, they could, but they chose not to, and it's their decision to make.
Other people might have made different judgement calls, but how can they be wrong about that? Do you think it's possible for anyone on the outside to prove that Mythos is safe? Seems like the people making claims it's somehow obviously safe so they're obviously lying aren't backing it up.
Also, Anthropic is not OpenAI. Sam Altman and Musk aren't writing Anthropic's press releases. Their lies aren't relevant when judging some other company's claims.
It's is sort of like saying, because there are lot of people on the Internet who lie, and you're a stranger on the Internet, you must also be lying. There's good reason to be skeptical of strangers in general, but going beyond that is guilt by association.
People can also reasonably disagree on what's going to happen with AI in the next few years. It's all speculation. But we do see fairly big improvements in AI every year. I think those articles you linked to are speculative but not nonsense, given the rather strange situation we are in.
-
Comment on The boy that cried Mythos in ~comp
skybrian (edited )Link ParentI'm somewhat hopeful that, eventually, we will have some core software components that simply don't have any bugs that anyone can find. In the meantime, there's going to be a lot of software updates.I'm somewhat hopeful that, eventually, we will have some core software components that simply don't have any bugs that anyone can find. In the meantime, there's going to be a lot of software updates.
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentWhat specifically are the extraordinary claims? The previous LLM's were already quite good at finding security issues and they're claiming the new one is better. They also have a track record of...What specifically are the extraordinary claims? The previous LLM's were already quite good at finding security issues and they're claiming the new one is better. They also have a track record of releasing increasingly better models.
It doesn't mean you can't be a little skeptical about new releases, but it doesn't add up to "wild bullshit" and "blatantly lying." Maybe you should substantiate those claims?
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentCould you share examples of prompts you use? I'm curious about what it would find on my hobby projects.Could you share examples of prompts you use? I'm curious about what it would find on my hobby projects.
-
Comment on The boy that cried Mythos in ~comp
skybrian (edited )Link ParentIt’s certainly a rather odd situation that these companies put out so many different kinds of warnings about their products, but dismissing it as “marketing” is rather cynical. In a way it’s the...It’s certainly a rather odd situation that these companies put out so many different kinds of warnings about their products, but dismissing it as “marketing” is rather cynical.
In a way it’s the opposite of marketing because it’s poisoning the well. Anti-AI sentiment among the general public is on the rise and the companies themselves are feeding it. It’s as if tobacco companies in the 1950’s were warning that their product causes cancer, or early car companies were predicting traffic jams and smog and urban sprawl. (To be clear, this is an imperfect analogy - there are ways to use AI that aren’t particularly harmful.)
It’s also a double bind where the cynics get to be cynical either way. If a company warned about safety then it’s fear-mongering and if they didn’t warn about it then it would be seen as covering up safety issues they knew about, like what the tobacco companies actually were doing in the 1950’s. Or more recently, Facebook tends to downplay safety concerns about social media, and look where that ended up.
But more fundamentally, the issue isn’t whether they warn or don’t warn, it’s that the product is being used in harmful ways and the public knows it. A company putting out a harmful product is going to be unpopular and warnings don’t help much. It’s like, if it’s so bad why are you selling it then? Warning people is good, but it doesn’t change the product.
I think it’s sensible to encourage companies to delay or cancel product releases if they seem dangerous or they have concerns about how it will be used. The idea that you have to prove the product is harmful to justify not releasing it is backwards. Instead you have to be sure it’s safe, or safe enough. So for example, I’m happy that Waymo has taken their time scaling up deployment of driverless cars. If the AI companies started being more cautious, that would probably be for the best. Unfortunately it’s effectively an arms race so it’s hard to slow it down.
It’s kind of weird to see a company being attacked for being too cautious.
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentThese supposed proofs that they're lying look to me more like complaints that Anthropic didn't publish detailed information. The "system card" is not a scientific paper and doesn't include the...These supposed proofs that they're lying look to me more like complaints that Anthropic didn't publish detailed information. The "system card" is not a scientific paper and doesn't include the data you'd need to prove things to a skeptic. It's unfortunate that we're taking their word on some things, but we'll just have to wait.
The 90-day public report does not exist yet, so I’m perhaps jumping ahead [...]
Yes, he is.
We then find a total of 250 runs: five trials per category, fifty categories. Wait, what? Who set up this test? AFL does that many mutation cases in a millisecond.
That's just dumb, fuzzing is not the same kind of test. They're different tools.
Anthropic is admitting, in their own footnote, that Sonnet 4.6 has the same triage ability as Mythos. Sonnet sees the same two “obvious” bugs. It just cannot close the exploitation step. Mythos’s entire frontier advantage over the prior model is therefore bupkis
"Just" is doing a lot of work there. Seems like a model that can find an exploit is more dangerous to release to the public than one that can find a bug?
I could go on, but this guy obviously has an axe to grind. He's going to find what he's looking for whether it's warranted or not.
-
Comment on ‘It’s shameful’: New York’s elite lash out at Zohran Mamdani’s second-home tax in ~finance
skybrian Link ParentThey had expectations about how much it would cost them. Now, there's an tax increase targeted specifically at them, so it will cost more. I don't think it's that hard to understand why some might...They had expectations about how much it would cost them. Now, there's an tax increase targeted specifically at them, so it will cost more. I don't think it's that hard to understand why some might think that's unfair? NYC is changing the deal, and the politics around it are such that it seems punitive. (Of course, that's why a lot of other people like it.)
New buyers will know going in what it costs.
-
Comment on The boy that cried Mythos in ~comp
skybrian LinkThe people who want to be dismissive are going to find reasons, but the security experts I follow are taking the AI-based security threat very seriously. Some guy wants to make people hate...The people who want to be dismissive are going to find reasons, but the security experts I follow are taking the AI-based security threat very seriously.
Some guy wants to make people hate Anthropic and I don't really see the point of writing a long takedown about it. I don't think we need to take a position on how much better Mythos is, because it doesn't really matter. There are more high-quality security bugs being found through a variety of AI-enabled means. The people who maintain important systems have a lot more work to do lately.
-
Comment on The boy that cried Mythos in ~comp
skybrian Link ParentFrom OpenAI's 2019 announcement about GPT-2: Seems to me that holds up well?From OpenAI's 2019 announcement about GPT-2:
These findings, combined with earlier results on synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns. The public at large will need to become more skeptical of text they find online, just as the “deep fakes(opens in a new window)” phenomenon calls for more skepticism about images.
Today, malicious actors—some of which are political in nature—have already begun to target the shared online commons, using things like(opens in a new window) “robotic tools, fake accounts and dedicated teams to troll individuals with hateful commentary or smears that make them afraid to speak, or difficult to be heard or believed.”
Seems to me that holds up well?
-
Comment on “Rediscovering” the operating system (AKA: the desktop is the killer app) in ~tech
skybrian LinkThis sounds more like rediscovering filesystems. I think naming and arranging files in folders is something a lot of people have gotten away from and don't want to go back to. An app could do that...This sounds more like rediscovering filesystems. I think naming and arranging files in folders is something a lot of people have gotten away from and don't want to go back to. An app could do that for you, though, like iTunes does.
The files are still there, unless it's a Sqlite database or something like that.
-
Comment on Adversaries leverage AI for vulnerability exploitation, augmented operations, and initial access in ~comp
skybrian LinkFrom the article: [...] [...]From the article:
- Vulnerability Discovery and Exploit Generation: For the first time, GTIG has identified a threat actor using a zero-day exploit that we believe was developed with AI. The criminal threat actor planned to use it in a mass exploitation event but our proactive counter discovery may have prevented its use. Threat actors associated with the People’s Republic of China (PRC) and the Democratic People's Republic of Korea (DPRK) have also demonstrated significant interest in capitalizing on AI for vulnerability discovery.
[...]
-
AI-Augmented Research and IO: Adversaries continue to leverage AI as a high speed research assistant for attack lifecycle support, while shifting toward agentic workflows to operationalize autonomous attack frameworks. In information operations (IO) campaigns, these tools facilitate the fabrication of digital consensus by generating synthetic media and deepfake content at scale, exemplified by the pro-Russia IO campaign “Operation Overload.”
-
Obfuscated LLM Access: Threat actors now pursue anonymized, premium tier access to models through professionalized middleware and automated registration pipelines to illicitly bypass usage limits. This infrastructure enables large scale misuse of services while subsidizing operations through trial abuse and programmatic account cycling.
[...]
The compromise of LiteLLM, an AI gateway utility for integrating multiple LLM providers is noteworthy. It highlights the expanding attack surface of AI platforms and the potential for impact across the software supply chain. Given the package's widespread use, this incident could lead to considerable exposure of AI API secrets from affected victims, which could be used to gain further access to systems for traditional intrusion operations.
There’s a bit more about it (and alternatives) in this article.