It’s about how tech billionaires have developed a religion based on poorly-interpreted science fiction, poorly-understood cognitive psychology, thanatophobia, and the need to feel morally justified about making unreasonable amounts of money.
The end result is that the world’s richest and most influential people are working for a future in which AI Jesus solves all our problems and all the matter in the universe is turned into computers for simulating happy humans.
Becker does an excellent job of breaking down the inaccurate scientific, mathematical, and moral beliefs underlying these goals. Nanotechnology and neuroscience don’t work the way these groups think they do. Neither does space travel.
EA philanthropists, when helping current real people, often take into account whether they’re from an ethnic group that they think is more likely to contribute to AI alignment.
Jeffrey Epstein is involved, of course.
you can see it in the redirection of far too many societal resources to solving a problem with a technology that’s probably not possible, for a future that’s probably not possible—and away from real and higher-probability existential problems like climate change, pandemics, or even cometary impacts.
Becker points out that humans have a long history of counterproductive, septic belief sets. The difference here is that billionaires have way, way too much power to act on those beliefs, and to push others to act on them as well. His proposed solution: tax billionaires out of existence
Since I'm a serendipity attractor, I'm actually a hair more than halfway through rereading Vernor Vinge's Zones of Thought trilogy right now. Highly recommended reading, both thought-provoking and...
Since I'm a serendipity attractor, I'm actually a hair more than halfway through rereading Vernor Vinge's Zones of Thought trilogy right now. Highly recommended reading, both thought-provoking and fine space opera. I don't think Emrys is overstating Vinge's prescience about modern information technology - she touches on AI singularities, alignment problems, propaganda (the "Net of a Million Lies"), network subversion, malware, digital immortality, civilizational collapse due to unmanageable complexity, longtermism, and a host of other currently pertinent issues in a book written in 1992.
As to whether Emrys exaggerates about EA advocates proposing logical extensions of "Earning to Give" that encompass extreme harms for greater gains, consider this example. The superficially rational framework of analysis and the fuzziness of the underlying data make all kinds of harmful activity plausible if viewed through the lens of "I might eventually do more good than harm." The moral calculus gets even more fragile if, as Emrys' essay mentions, you can include hypothetical good done for innumerable hypothetical future people. I've read a moderate amount of EA and Rationalist discussion, and the level of self-justifying abstraction by people immersed in the community can be shocking.
There's a recent interview with William McAskill where he acknowledges the hazards of that moral balancing act and clarifies his position in light of the Bankman-Fried fraud. But his system of civilizational risk priorities still devalues known harms to currently existing humans in favor of abstract risks whose probabilities are unfalsifiable because Big Scary Machine God.
Note: If you haven't read Ruthanna Emrys' A Half-Built Garden, it's a nice example of the "hopepunk" SF sub-genre.
Well, I agree that there’s some crazy stuff out there. You know how philosophers will sometimes come up with edgy thought experiments like trolley problems or radical skepticism? There are quite a...
Well, I agree that there’s some crazy stuff out there.
You know how philosophers will sometimes come up with edgy thought experiments like trolley problems or radical skepticism? There are quite a lot of philosophical discussions happening in LessWrong and other forums.
I don’t usually consider this a problem, any more than taking a philosophy course in college or college freshman having late night philosophical discussions is a problem. Often it’s “insight porn” that has no practical consequences.
But it’s true that it could be a problem when someone takes it too seriously. Ideas can be influential.
Peter Singer is an example of a notorious philosopher who has come up with influential ideas both good and bad. He has argued for vegetarianism and the moral imperative of donating to the poor, both of which are influential in EA circles. He also infamously argued that killing severely developmentally deformed babies is okay, which is the opposite of influential in that basically nobody agrees.
And it’s worth pointing out that William McAskill is also a philosopher.
It seems like philosophers and would-be philosophers have a lot to answer for since sometimes their ideas are bad, but that putting an idea out there is not the same as taking it seriously and deciding to act on it, and that the community shouldn’t need to answer for every bad idea that someone posts. (Criticism of the actual arguments is of course fine.)
But we should pay attention to what some people actually do. For example, there have been some dangerous cults. It’s unclear what can be done to prevent that.
I do think the very long term science fictional thinking that you see sometimes is an unfortunate distraction. For example, advocating for colonizing Mars as a “backup” for Earth. Why should I care about people on Mars if everyone dies on Earth? That’s a catastrophe either way. Similarly for far-future potential people that we will never meet. They aren’t real and their imagined fate should carry very little weight.
What distinguishes Effective Altruism as a philosophy is that it's given lip-service by extremely influential billionaires. McAskill's Center for Effective Altruism partook of their "charitable"...
What distinguishes Effective Altruism as a philosophy is that it's given lip-service by extremely influential billionaires. McAskill's Center for Effective Altruism partook of their "charitable" donations and influence to its great detriment. We also now know where Elon Musk's claimed alignment with Effective Altruism philosophy has led. Anthropic's reputation is gilded by its founders' and employees' continuing claims about adherence to EA-inspired philanthropy.
I don't know if philosophers can be held generally responsible for what adherents do with their ideas, but history seems to show that they're not always innocent in engaging with them.
Yes, but I think that needs to be judged on a case-by-case basis. Science fiction and fantasy writers are in a similar position. For example, I don't think Neal Stephenson is responsible because...
Yes, but I think that needs to be judged on a case-by-case basis. Science fiction and fantasy writers are in a similar position. For example, I don't think Neal Stephenson is responsible because Zuckerberg thought building a metaverse would be cool. If you write a story and it "goes viral," so to speak, then lots of people will read it and you have no control over what powerful people do with your ideas. You just do the best you can under the circumstances.
Influential charities are in a tricky position: when some billionaire offers you huge amount of money for what you believe is a worthy cause, should you take the money? Arguably that money is better off in your hands than whatever else they might have done with it, but the downside is that you're now associated with someone who might turn out to be Bankman-Fried. Even if there are formally no strings attached, there is the reputational hit.
Also, the trouble with judging on a case-by-case basis is that then you have to.investigate each case, and that results in doing a lot of homework. Who has the time? If you're lucky then a trustworthy journalist will do it for you. But not every writer is trustworthy on every subject.
And so we often go by reputation and rumor, copying our homework from other people. I think that often results in injustice.
And that's why I often go with "I'm out of touch and don't know anything about it" as a default opinion.
This is someone who is clearly willing to make shit up about people they dislike. I doubt they could find even one post where someone in EA advocates for making money by going into weapons...
The seemingly sensible goal sometimes encompassed the belief that it’s morally better to go into a lucrative-but-harmful field (e.g., weapons dealing) where you can contribute more than a directly helpful one that pays poorly.
This is someone who is clearly willing to make shit up about people they dislike. I doubt they could find even one post where someone in EA advocates for making money by going into weapons dealing.
I mean, it's an extreme example, and you are right to call it out cause it doesn't really make sense, but that is the logic. It's ok to ruin the worlds climate with data centers as long as the...
I mean, it's an extreme example, and you are right to call it out cause it doesn't really make sense, but that is the logic. It's ok to ruin the worlds climate with data centers as long as the computer solves the problems for humanity.
I don’t see how that has anything to do with young people deciding what sort of career to pursue, which is what the “earn to give” stuff is about. If someone decides that it’s okay to become a...
I don’t see how that has anything to do with young people deciding what sort of career to pursue, which is what the “earn to give” stuff is about. If someone decides that it’s okay to become a software engineer and they’ll try to do good by giving more to charity, that doesn’t make them responsible for how many data centers are built. (I suppose it could happen, but it’s a rather unlikely career trajectory.)
Since I'm a serendipity attractor, I'm actually a hair more than halfway through rereading Vernor Vinge's Zones of Thought trilogy right now. Highly recommended reading, both thought-provoking and fine space opera. I don't think Emrys is overstating Vinge's prescience about modern information technology - she touches on AI singularities, alignment problems, propaganda (the "Net of a Million Lies"), network subversion, malware, digital immortality, civilizational collapse due to unmanageable complexity, longtermism, and a host of other currently pertinent issues in a book written in 1992.
As to whether Emrys exaggerates about EA advocates proposing logical extensions of "Earning to Give" that encompass extreme harms for greater gains, consider this example. The superficially rational framework of analysis and the fuzziness of the underlying data make all kinds of harmful activity plausible if viewed through the lens of "I might eventually do more good than harm." The moral calculus gets even more fragile if, as Emrys' essay mentions, you can include hypothetical good done for innumerable hypothetical future people. I've read a moderate amount of EA and Rationalist discussion, and the level of self-justifying abstraction by people immersed in the community can be shocking.
There's a recent interview with William McAskill where he acknowledges the hazards of that moral balancing act and clarifies his position in light of the Bankman-Fried fraud. But his system of civilizational risk priorities still devalues known harms to currently existing humans in favor of abstract risks whose probabilities are unfalsifiable because Big Scary Machine God.
Note: If you haven't read Ruthanna Emrys' A Half-Built Garden, it's a nice example of the "hopepunk" SF sub-genre.
Well, I agree that there’s some crazy stuff out there.
You know how philosophers will sometimes come up with edgy thought experiments like trolley problems or radical skepticism? There are quite a lot of philosophical discussions happening in LessWrong and other forums.
I don’t usually consider this a problem, any more than taking a philosophy course in college or college freshman having late night philosophical discussions is a problem. Often it’s “insight porn” that has no practical consequences.
But it’s true that it could be a problem when someone takes it too seriously. Ideas can be influential.
Peter Singer is an example of a notorious philosopher who has come up with influential ideas both good and bad. He has argued for vegetarianism and the moral imperative of donating to the poor, both of which are influential in EA circles. He also infamously argued that killing severely developmentally deformed babies is okay, which is the opposite of influential in that basically nobody agrees.
And it’s worth pointing out that William McAskill is also a philosopher.
It seems like philosophers and would-be philosophers have a lot to answer for since sometimes their ideas are bad, but that putting an idea out there is not the same as taking it seriously and deciding to act on it, and that the community shouldn’t need to answer for every bad idea that someone posts. (Criticism of the actual arguments is of course fine.)
But we should pay attention to what some people actually do. For example, there have been some dangerous cults. It’s unclear what can be done to prevent that.
I do think the very long term science fictional thinking that you see sometimes is an unfortunate distraction. For example, advocating for colonizing Mars as a “backup” for Earth. Why should I care about people on Mars if everyone dies on Earth? That’s a catastrophe either way. Similarly for far-future potential people that we will never meet. They aren’t real and their imagined fate should carry very little weight.
What distinguishes Effective Altruism as a philosophy is that it's given lip-service by extremely influential billionaires. McAskill's Center for Effective Altruism partook of their "charitable" donations and influence to its great detriment. We also now know where Elon Musk's claimed alignment with Effective Altruism philosophy has led. Anthropic's reputation is gilded by its founders' and employees' continuing claims about adherence to EA-inspired philanthropy.
I don't know if philosophers can be held generally responsible for what adherents do with their ideas, but history seems to show that they're not always innocent in engaging with them.
Yes, but I think that needs to be judged on a case-by-case basis. Science fiction and fantasy writers are in a similar position. For example, I don't think Neal Stephenson is responsible because Zuckerberg thought building a metaverse would be cool. If you write a story and it "goes viral," so to speak, then lots of people will read it and you have no control over what powerful people do with your ideas. You just do the best you can under the circumstances.
Influential charities are in a tricky position: when some billionaire offers you huge amount of money for what you believe is a worthy cause, should you take the money? Arguably that money is better off in your hands than whatever else they might have done with it, but the downside is that you're now associated with someone who might turn out to be Bankman-Fried. Even if there are formally no strings attached, there is the reputational hit.
Also, the trouble with judging on a case-by-case basis is that then you have to.investigate each case, and that results in doing a lot of homework. Who has the time? If you're lucky then a trustworthy journalist will do it for you. But not every writer is trustworthy on every subject.
And so we often go by reputation and rumor, copying our homework from other people. I think that often results in injustice.
And that's why I often go with "I'm out of touch and don't know anything about it" as a default opinion.
This is someone who is clearly willing to make shit up about people they dislike. I doubt they could find even one post where someone in EA advocates for making money by going into weapons dealing.
Don't get your news from people like that.
I mean, it's an extreme example, and you are right to call it out cause it doesn't really make sense, but that is the logic. It's ok to ruin the worlds climate with data centers as long as the computer solves the problems for humanity.
I don’t see how that has anything to do with young people deciding what sort of career to pursue, which is what the “earn to give” stuff is about. If someone decides that it’s okay to become a software engineer and they’ll try to do good by giving more to charity, that doesn’t make them responsible for how many data centers are built. (I suppose it could happen, but it’s a rather unlikely career trajectory.)