This is one of Eliezer's worse essays, in my opinion, which I say while both having a high opinion of him and completely agreeing with the premise. (Basically: we shouldn't firebomb Sam Altman's...
This is one of Eliezer's worse essays, in my opinion, which I say while both having a high opinion of him and completely agreeing with the premise. (Basically: we shouldn't firebomb Sam Altman's house).
What I disagree with the most is his repeated framing of predictability as the thing that distinguishes legitimate state-violence from illegitimate vigilante violence. 1) much state violence is not predictable, and people can't always avoid violence just by following the law. Ask anyone who's experienced bad policing. 2) much vigilante violence is predictable, and that doesn't make it okay. 3) More to the point, if you believe that ASI will kill everyone, then legitimacy is probably completely irrelevant. In that case, the real argument should be that vigilante violence won't work. Eliezer does touch on this, but if that's what he's thinking the entire essay should be about that.
My guess would be that this essay falls into Yudkowsky's fatal flaw. His self-image is "guy with unique thoughts." (To be fair to him, he's often right about that.) To remain true to that self-image, he tries to take basically everything from first principles. Sometimes that leads to genuinely novel insights, but more often it ends up like this - where there's already a billion pages of political philosophy exploring the differences between state and nonstate violence, and this new argument ended up being both unnecessary and distracting from his main points.
Maybe I would need to dig deeper to be more sure of this, but just from the content of the essay, I would assume that Yudkowsky agrees or mostly agrees with all three of your numbered points,...
Maybe I would need to dig deeper to be more sure of this, but just from the content of the essay, I would assume that Yudkowsky agrees or mostly agrees with all three of your numbered points, though I'm less sure on 3).
I don't even think he fully went all the way to designate state violence as legitimate if predictable, but simply named predictability as one possible prerequisite and as making a moral difference.
Also, we can't look into the insides of the guy, but I reckon there's something true to your last paragraph, well put.
Yeah, I also think Yudkowsky would agree with those points. Whatever his faults, he's an intensely moral and thoughtful guy. I just don't think he argued them very well, if that makes sense.
Maybe I would need to dig deeper to be more sure of this, but just from the content of the essay, I would assume that Yudkowsky agrees or mostly agrees with all three of your numbered points, though I'm less sure on 3).
Yeah, I also think Yudkowsky would agree with those points. Whatever his faults, he's an intensely moral and thoughtful guy. I just don't think he argued them very well, if that makes sense.
For people who heard something about Yudkowsky calling for airstrikes on datacenters, this article seems like a useful clarification: … …
For people who heard something about Yudkowsky calling for airstrikes on datacenters, this article seems like a useful clarification:
If an ASI ban is to accomplish anything at all, it has to be effective everywhere.
…
ASI is a product that kills people standing on the other side of the planet. Driving an AI company out of just your own city will not protect your family from death. It won't even protect your city from job losses, earlier in the timeline.
And similarly: To impede one executive, one researcher, or one company, does not change where AI is heading.
…
Even if you're desperate, an outburst of violence usually will not actually solve your problems!
This is one of Eliezer's worse essays, in my opinion, which I say while both having a high opinion of him and completely agreeing with the premise. (Basically: we shouldn't firebomb Sam Altman's house).
What I disagree with the most is his repeated framing of predictability as the thing that distinguishes legitimate state-violence from illegitimate vigilante violence. 1) much state violence is not predictable, and people can't always avoid violence just by following the law. Ask anyone who's experienced bad policing. 2) much vigilante violence is predictable, and that doesn't make it okay. 3) More to the point, if you believe that ASI will kill everyone, then legitimacy is probably completely irrelevant. In that case, the real argument should be that vigilante violence won't work. Eliezer does touch on this, but if that's what he's thinking the entire essay should be about that.
My guess would be that this essay falls into Yudkowsky's fatal flaw. His self-image is "guy with unique thoughts." (To be fair to him, he's often right about that.) To remain true to that self-image, he tries to take basically everything from first principles. Sometimes that leads to genuinely novel insights, but more often it ends up like this - where there's already a billion pages of political philosophy exploring the differences between state and nonstate violence, and this new argument ended up being both unnecessary and distracting from his main points.
Maybe I would need to dig deeper to be more sure of this, but just from the content of the essay, I would assume that Yudkowsky agrees or mostly agrees with all three of your numbered points, though I'm less sure on 3).
I don't even think he fully went all the way to designate state violence as legitimate if predictable, but simply named predictability as one possible prerequisite and as making a moral difference.
Also, we can't look into the insides of the guy, but I reckon there's something true to your last paragraph, well put.
Yeah, I also think Yudkowsky would agree with those points. Whatever his faults, he's an intensely moral and thoughtful guy. I just don't think he argued them very well, if that makes sense.
For people who heard something about Yudkowsky calling for airstrikes on datacenters, this article seems like a useful clarification:
…
…