skybrian's recent activity
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian (edited )Link ParentOkay but Chinese AI firms aren’t much like poor people sleeping under bridges. They seem to be be doing fine despite US export restrictions. The alternative to some kind of negotiated AI slowdown...Okay but Chinese AI firms aren’t much like poor people sleeping under bridges. They seem to be be doing fine despite US export restrictions.
The alternative to some kind of negotiated AI slowdown is unrestricted capitalism. The industry just keeps going as fast as possible despite all the dangers and downsides. Maybe, as with climate change, someone should try to do something about it?
If fear of the Chinese is what gets the US to the negotiating table then maybe that’s not a bad thing?
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian Link ParentSure, past performance is no guarantee of anything. But if AI peaks right away, that would be quite a coincidence. There’s new research being published all the time and it would be weird if none...Sure, past performance is no guarantee of anything. But if AI peaks right away, that would be quite a coincidence. There’s new research being published all the time and it would be weird if none of it worked out in practice.
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian Link ParentIs that based on anything other than being attracted to conspiracy theories? There’s not much in the article about how this letter came about, but it seems to be worker-led rather than a top-down...Is that based on anything other than being attracted to conspiracy theories? There’s not much in the article about how this letter came about, but it seems to be worker-led rather than a top-down effort:
ChatGPT-maker OpenAI and its main rival, Anthropic, which provides the Claude chatbot, endorsed a petition released Tuesday that had been signed by hundreds of employees at the two companies, as well as Google, Meta and other leading companies working on AI.
The petition itself has this at the bottom:
“This statement was published by employees of frontier AI companies, with organizational support from two independent nonprofits, Guidelight AI Standards and Encode AI.”
Guideline AI Standards seems to be a non-profit started by ex-OpenAI workers:
Guidelight is a new, independent nonprofit, founded by Page Hedley and Steven Adler (both ex-OpenAI safety leaders), that works with independent experts, AI companies, and the public to define important safety practices and promote their adoption.
Looks like Encode AI was previously Encode Justice and started as a youth activist movement to defeat California Proposition 25:
Revanur’s interest in AI regulation began in 2020, when she founded Encode Justice, a youth-led, AI-focused civil-society group, to mobilize younger generations in her home state of California against Proposition 25, a ballot measure that aimed to replace cash bail with a risk-based algorithm. After the initiative was defeated, the group kept on, focusing on educating and mobilizing peers around AI policy advocacy. The movement now counts 800 young members in 30 countries around the world, and has drawn comparisons to the youth-led climate and gun-control movements that preceded it.
Employees and ex-employees of AI labs have their own political opinions and they can organize, with help from people outside. Let’s not deny the agency of the people involved.
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian Link ParentI also suspect that there’s plenty of room for algorithmic improvements and that does seem to be the trend, but LLM’s don’t work much like the human brain so that doesn’t seem like a relevant...I also suspect that there’s plenty of room for algorithmic improvements and that does seem to be the trend, but LLM’s don’t work much like the human brain so that doesn’t seem like a relevant comparison.
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian Link ParentExcept, nothing else that we’ve seen this year suggests that AI is plateauing. Better models are being released all the time.Except, nothing else that we’ve seen this year suggests that AI is plateauing. Better models are being released all the time.
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian Link ParentThey are requesting that the "U.S. government support an international effort." Presumably the goal is to make an agreement so that everyone abides by the same rules. If the increase in...They are requesting that the "U.S. government support an international effort." Presumably the goal is to make an agreement so that everyone abides by the same rules.
If the increase in "intelligence" (whatever that means) were capped, I imagine there would still be competition in other dimensions like price, latency, writing style, and so on.
-
Comment on OpenAI and Anthropic endorse call for US government to "pace" AI progress in ~comp
skybrian LinkFrom the article: [...] The way I’m thinking about it is that this is an attempt to agree on a new Moore’s law.From the article:
ChatGPT-maker OpenAI and its main rival, Anthropic, which provides the Claude chatbot, endorsed a petition released Tuesday that had been signed by hundreds of employees at the two companies, as well as Google, Meta and other leading companies working on AI.
The letter warned that companies could be close to automating the work of AI research, a development that it said could lead the technology to careen out of control.
“We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development,” the petition said. (The Washington Post has a content partnership with OpenAI.)
Coordinating international agreement to limit AI development would be a gargantuan diplomatic task that has been likened to implementing a nuclear weapons control treaty for software that can be freely distributed through the internet.
[...]
The new petition is different because it has won corporate endorsements for a more specific government intervention on the industry, albeit at an unknown point in the future.
Its signatories also include senior executives from OpenAI, Meta and Google, companies that have generally been much more opposed to strict AI regulation than Anthropic.
The way I’m thinking about it is that this is an attempt to agree on a new Moore’s law.
-
OpenAI and Anthropic endorse call for US government to "pace" AI progress
16 votes -
Comment on Some thoughts about Anthropic’s new cryptanalysis results in ~comp
skybrian LinkFrom the article: [...]From the article:
As a researcher I’ve also been spending a lot of time with models, talking through various ideas. I don’t think I will surprise anyone when I say that they’re obviously getting better, even over the course of the past few months. While I don’t have Mythos and $100k to spend, I have been able to query at least one new advanced unreleased model, and I also have received some surprising new “results” to questions that I’ve been interested in for a few years.
Which brings me to the real problem: just because a model spits out an apparent new result, this does not mean the result is real. Even if models are good at producing real results, they’re much better at producing results that look real but are misleading. This can be enormously frustrating, and often means that human attention is more necessary than ever.
There are exceptions to this rule: for “full” attacks like HAWK, where the attack runs in a few hours (against a weaker version of the scheme), verification is extremely easy. You can just send over the code and let anyone check that it recovers keys and signs real chosen messages. For more subtle speedup attacks like the AES result, checking validity is not so easy. Here the approach is more specific: formally-verifiable Lean proofs can help here, but (even where these proofs are easy to make), such proofs are still highly sensitive to how you’ve formulated the theorem statement, and that often requires human experts to check.
You’ll probably notice that many of the exciting recent mathematical results have had this flavor: they either include a machine-checkable proof of a well-understood theorem, or (like the Jacobian conjecture) they involve finding a simple counterexample you can compute on. Alternatively, a bunch of experts spent a lot of time reviewing the result and were eventually convinced by it. This need for some humans to check the work is going to slow down our progress. For non-devastating examples of cryptanalysis, this is probably where we’re going to be for a while.
[...]
With that said, I said there was good news, and I meant it. Right now we’re in the midst of a historic transition from traditional public-key algorithms based on EC-based cryptography and RSA, moving over to new post-quantum algorithms based on novel problems. This is why there are so many standards like HAWK being considered. If there was ever a perfect time for a massive new public cryptanalysis capability to come on line, we’re in it. So unless AIs succeed in undermining all of our hard problems altogether (or we live in Impagliazzo’s Minicrypt) then this could not be a better time for AI to get good at cryptanalysis. In the best case, the result is that we gain real confidence in the problems we’ve identified, and the cryptanalysis literature gets a lot more robust. Hopefully.
For scientists: this is also a wonderful time. You now have a plastic pal who’s fun to be with, and you can talk over your hardest problems. At the same time it’s not yet smart enough that it can solve all of them without your assistance. And even better, the pace of new findings is speeding way up. This is mostly good! If you’re energetic. I still have many questions, like: “who should get credit for these new results” and “who will review all of these new results” but so far I’m not panicked. The world is getting modestly better. For now.
For the world: I don’t know. If you’re under the impression that these models are “glorified autocomplete” or that progress is slowing down, I need to urge you: stop thinking that. The models are very intelligent and capable, they are getting better at a fast clip. I can cite measurable and impressive progress over just the past five months on specific types of problem I’ve asked them to look at. If there’s a ceiling out there, I don’t yet see evidence of it. The people who think models are dumb are mostly using Google’s free AI search results, and not interacting with the high-end stuff (which only costs $20, so it’s not out of reach.) And they’re mostly not working in new areas.
On the other hand: if you think that models are super-intelligent or that AGI is already here, you should also stop thinking that. Working with these tools is like swimming in a pond where the ground drops off sharply. One minute you’re wading comfortably and there’s support under your feet. Then suddenly you cross a specific line, and you’re back to swimming on your own. This analogy is my best way to explain what it feels like when the model goes from helpful to clueless. Right now it’s easy for a human being to find that line if you’re doing advanced research, so you know where the intelligence drops off. But the line is moving. You can feel it slowly drifting outwards under your feet.
-
Some thoughts about Anthropic’s new cryptanalysis results
14 votes -
Comment on Curing concrete - improving the carbon costs of an essential material (gifted link) in ~enviro
skybrian LinkSide comment: that's an awful lot of tags! Are you autogenerating them somehow?Side comment: that's an awful lot of tags! Are you autogenerating them somehow?
-
Comment on Finding bugs in Raft implementations in ~comp
skybrian Link ParentYes, I also avoid testing against mocks when testing against a real implementation is practical. (Databases and browsers come to mind.) On the other hand, testing in a real environment won't tell...Yes, I also avoid testing against mocks when testing against a real implementation is practical. (Databases and browsers come to mind.)
On the other hand, testing in a real environment won't tell you whether your code is portable or conforms to a standard. Testing against multiple implementations is good for comparison. Maybe you could test against real browsers and also a theoretically ideal, abstract browser? That's where a proof might be useful, so you tell when your implementation is theoretically correct, but reality is letting you down.
-
Comment on US schools are adding pepper-spraying drones to help combat active shooters in ~society
skybrian Link Parent"Perception is reality" is overstating it. Yes, sometimes it happens that believing in something makes it true. This is why it's hard to beat the front-runner in an election. But after a market..."Perception is reality" is overstating it. Yes, sometimes it happens that believing in something makes it true. This is why it's hard to beat the front-runner in an election. But after a market crash, we discover that, while getting people to believe in things can indeed be quite powerful, it doesn't necessarily make them true.
Also, as every failed political candidate knows, getting millions of people to believe something is not that easy. Unless they already want to believe.
-
Comment on US schools are adding pepper-spraying drones to help combat active shooters in ~society
skybrian (edited )Link ParentWe are always changing society but sometimes it’s a side effect, which seems easier than a deliberate change. Side effects are like like water running downhill rather than having to pump it...We are always changing society but sometimes it’s a side effect, which seems easier than a deliberate change. Side effects are like like water running downhill rather than having to pump it uphill. Even for deliberate attempts at changes like ads, the ones that take advantage of existing weaknesses will be more effective, unfortunately.
If you have a drone, maybe you can get by without the cop? They seem like competing ways to spend money on security. Hiring people is expensive, too.
I can imagine a drone being marketed as a way to cut labor costs. Maybe it would be a higher up-front cost, but eventually pays for itself assuming there would be a cop otherwise.
Though of course they can do different things.
-
Comment on US schools are adding pepper-spraying drones to help combat active shooters in ~society
skybrian Link ParentI think the trouble is that we are guessing about root causes and how to address them. Well, there is gun control, but that’s politically very difficult. And if your plan is to first fix Congress...I think the trouble is that we are guessing about root causes and how to address them. Well, there is gun control, but that’s politically very difficult. And if your plan is to first fix Congress or fix society then the trouble is that it’s an “if everyone will just” sort of plan.
So, instead let’s go with the unproven techno-fix because that just involves spending money and doesn’t require changing American society. The world gets a little more sci-fi because just about anything is easier than changing society.
-
Comment on US Federal Communications Commission bans foreign-produced solar inverters in ~enviro
skybrian LinkFrom the article:From the article:
The Federal Communications Commission Public Safety and Homeland Security Bureau added foreign-produced power inverters to its Covered List, triggering an immediate and absolute ban on new equipment sales in the United States.
This means any solar or battery storage project relying on an inverter made outside the United States that has not already received an official FCC ID cannot legally turn on or interconnect. Because there is no phase-in period, grandfather clause, or grace window, the regulatory pipeline is frozen today, forcing developers to halt active procurements and find new hardware vendors.
The action effectively overrides the Department of Energy analysis from January 2026, which inspected 30 Chinese inverters and found zero evidence of malicious hardware. The White House interagency council determined that physical bugs do not matter because the risk is purely digital. The administration ruled that the wireless connectivity inherent in modern smart inverters allows foreign adversaries to push firmware updates that could shut down solar arrays remotely, making all foreign-assembled units an unacceptable threat to the critical power grid.
The immediate practical result is a massive equipment shortage that will delay upcoming commercial and utility projects. Department of Energy data shows that domestic manufacturers supply only seven percent of the U.S. solar inverter market, leaving a 93% deficit that cannot be filled by local factories anytime soon.
The hardware blockade hits right as developers plan to connect more than 58,000 MW of new solar and storage over the next year. Without certified inverters, fully built solar farms will sit dark and unable to feed electricity to the grid.
-
US Federal Communications Commission bans foreign-produced solar inverters
29 votes -
Comment on The apples and oranges tribunal in ~society
skybrian LinkFrom the article: [...] [...] [...] I'll add that market prices are often wrong (inconsistent). The market gives an answer, but nothing says it's a correct answer. It's what we have.From the article:
Suppose that apples sell for more than oranges and Parliament in [its] wisdom decides that, at last, apples and oranges must be compared. Not by shoppers — shoppers are biased, they merely reveal what they are willing to pay — but by a tribunal, which will determine whether apples and oranges are of truly equal value and thus must sell at the same price.
What would the tribunal need to know?
[...]
To determine the “just” price of apples and oranges, the tribunal would need the entire general-equilibrium system.
[...]
Britain is now running this experiment in the labor market–Is a retail worker equal to a warehouse worker? A canteen worker equal to a coal miner? A dinner lady equal to a gravedigger?
Under the Equality Act’s “equal value” provisions, tribunals compare jobs by scoring their intrinsic properties — effort, skill, responsibility, working conditions — the labor theory of value applied to labor. How is it going? The Tesco litigation began in 2018; the tribunal’s fact-finding hearing ran 36 days, its judgments run to more than 900 pages resting on some 19,000 pages of training manuals, and the independent experts have yet to begin the report that will actually say whether a shelf-stacker’s job equals a warehouse worker’s. Eight years, and the calculation has not started. Apples and oranges, adjudicated but not, as Orwell or Marx or Stafford Beer might have imagined, by a industrial bureaucracy or by an all-knowing artificial intelligence but by lawyers and commissions and tribunals. The worst of all worlds.
And having discovered that the tribunal cannot price two jobs in a decade, the government now proposes to add race and disability comparisons and an enforcement unit to publish official guidance on which reasons for a wage difference are permissible. A bureau of allowable scarcities.
[...]
The market compares apples and oranges every day. It is the only institution that can. But there is a deeper error here than computation. Suppose the tribunal succeeded. Suppose that after another decade it delivered the true and final score, shelf-stacker versus warehouseman. What would it have found? Not justice. A wage is not a grade on your character or a measure of your worth as a human being. A wage is a price — a report on how scarce your skills are relative to the desires of people you will never meet. Nurses are not morally less worthy than plumbers should they earn less than plumbers or vice-versa, and no one thinks otherwise except the tribunals.
I'll add that market prices are often wrong (inconsistent). The market gives an answer, but nothing says it's a correct answer. It's what we have.
-
The apples and oranges tribunal
4 votes
Yes, there are valid reasons for it, but the result is reflexive populist doomerism. People end up taking the same side as hard-core libertarians because they can't see how regulation might be a good thing. If people in the industry are for regulation and international agreements then they must be a conspiracy.
It's kind of like horseshoe theory.