I heard about this case from a family member and it sadly rings relatable. I was in circles with at least one individual for a fairly long period of time who sounds exactly like the kind of person...
I heard about this case from a family member and it sadly rings relatable. I was in circles with at least one individual for a fairly long period of time who sounds exactly like the kind of person who would unabashedly support this.
It's odd just how these similar behaviors/fields/beliefs overlap into this kind of group, but it's not a total anomaly other than actually acting on the threats of violence.
Related AP article: A timeline of activities of a cultlike group tied to the killing of a Border Patrol agent Edit: Longer articles with more details and an interactive timeline from SF Chronicle:...
This is related and I found it very interesting: https://aiascendant.substack.com/p/extropias-children Chapter 2 covers Ziz specifically, but a broader picture of rationalism is probably helpful here
This is a great resource. Thank you. It makes sense to me that the worldview or subculture that gets called "rationalism" by some Bay Area tech people (not to be confused with the older, more...
This is a great resource. Thank you.
It makes sense to me that the worldview or subculture that gets called "rationalism" by some Bay Area tech people (not to be confused with the older, more technical, and more specific point of view called "rationalism" from academic philosophy) would be correlated with psychosis. The way people think and write and speak and interact in that world seems almost like it's engineered to make you go insane.
For some reason, my mind is connecting the way that "rationalists" talk to the way that people who are emotionally abusive in intimate relationships talk. I don't know if I can really articulate the connection. I think there is some similarity in terms of talking to you in a way that undermines your trust in yourself — in your ability to reason, your perception, your intelligence, your sanity — and that undermines your trust in and connection to other people.
In an abusive relationship, a person who is abusing their partner will often try to isolate them from their friends and family. "Rationalists" promote the idea that people who don't think like them and believe different from them are less rational, less intelligent, less capable, less competent, and not to be trusted. They promote a feeling of alienation from mainstream society. I can see how that would make a person feel less and less grounded over time the more they are exposed to "rationalist" thought and the more they are immersed in the culture. I can see how, especially if they have other risk factors, it could make a person more susceptible to psychosis.
Now that I'm thinking about it, there's also a more one-to-one correlation, in that many "rationalists" do use bullying tactics and are simply just directly verbally abusive to outsiders and each other. That isn't so unusual for large social platforms like Reddit and Twitter (not that being usual makes it good, safe, or healthy), but it feels more sinister for it to be so common and so normalized in a smaller and more personal online community. (It's also modelled by "leaders"/prominent people in the community).
(By the way, I'm putting "rationalism" and "rationalists" in scare quotes because I just find it so obnoxious and annoying how they see themselves as exemplifying superior rationality and I want to try to give less power to that branding manoeuvre.)
I want to give a citation for my claim that people in this community "use bullying tactics and are simply just directly verbally abusive to outsiders and each other." I was looking up something...
I want to give a citation for my claim that people in this community "use bullying tactics and are simply just directly verbally abusive to outsiders and each other."
I was looking up something completely different and stumbled upon this comment about one of the most prominent members of the Bay Area "rationalist" community, who is the president of the Machine Intelligence Research Institute (MIRI). This comment comes from someone who used to work at MIRI:
I wish some people at MIRI had explicitly told me up front something like, “Hey, if you mess up a lunch order, you may want to avoid Nate until the next day. He is a very high-performing researcher, and you should not expect typical levels of patience or anger management from him. Also, if you try to stand up for yourself, he may simply cut you off and storm out of the room. Such is the price of having world-savers…do you have thick enough skin to work here?”
What the former employee continues with is kind of sad:
And I would have said, “Ah, I appreciate the candor and respect. Seems like you guys are making a reasonable tradeoff--after all Newton was notoriously prickly too. But I’m probably a bad fit for the role. Thanks for your consideration, and I look forward to seeing y’all at the next house party :]”
That feels... disturbing?
The former employee tells a few stories about the MIRI president's behaviour, such as:
...one day Nate got a flat tire out on the road, which is dangerous, and he was understandably upset. He complained to my manager who informed me. I saw Nate in the office kitchen later that day (a Saturday) and thought it was an appropriate time to bring up again that I was having trouble with our available pump. I didn’t know how to–“Learn!” he snapped and then stormed out of the room.
And:
He got really angry at me when the rest of the office outvoted him on the choice of lunch catering. He had veto power on restaurant selection, and that was fine with me, but the anger was apparently not helping him remember to use it when it mattered.
That's demented.
There are other comments on the same post where people say they've experienced similar behaviour from Nate. A woman who was dating Nate at the time the post was published wrote:
I often find that in disputes I feel dismissed, I perceive him as having a significant lack of curiosity about my worldview (and believe he's explicitly said he's not curious about perspectives he anticipates to have no value to him).
Iirc he's explicitly said he doesn't respect my thinking (edit: he clarifies he respects it in some areas but not others), and from my perspective this radiates off him whenever we fight. I often feel like I have trouble trusting my own mind, I doubt myself, and despite my best attempts I somehow come out of disputes thinking I must be the one who's wrong. It's weird to have a partner who's so shockingly good in so many ways, yet we have maybe the worst fights I've ever experienced in romantic relationships. (Though he says other girls he's dated don't have this problem and I am unusual)
Tellingly, a lot of the comments read to me as variously minimizing, excusing, justifying, or rationalizing this kind of behaviour, using euphemistic language, arguing that people should allowed to behave and communicate in ways that are outside the norm, saying that it must be frustrating for Nate to work with people who aren't as smart as him, etc.
This all lines up with everything else I know about the community norms. Cruelty and emotional violence are largely accepted and not challenged nearly enough.
I worked with Nate quite some time ago now, trying to improve the operations of MIRI. One thing has stayed in my verbal repertoire from my short time working there: "You can't just 'be smart' at...
I worked with Nate quite some time ago now, trying to improve the operations of MIRI. One thing has stayed in my verbal repertoire from my short time working there: "You can't just 'be smart' at it." He had apparently significant domain expertise, but when it came to anything outside of that he was quick, confident, and wrong. Whenever any system was "irrational", it was wrong and it chafed at him to obey the rules. There was no space for just following payroll or administrative law because it was the law, it had to be explained to him until it made sense or until he threw up his hands and gave up fighting it. It was an incredibly exhausting work environment, and I gave up and got away from him very, very quickly.
When I hear stories like this, I can't help but think that a person who putatively has a high level of competence in some technical domain would increase their competence if they got a grip on...
When I hear stories like this, I can't help but think that a person who putatively has a high level of competence in some technical domain would increase their competence if they got a grip on some of the fundamental human, emotional things, like integrating information and perspectives from other people, the willingness to let go of being right or proving your point, and some kind of balance between or integration of self-doubt and self-confidence. If you're fuming to the point of making other people uneasy or even lashing out at someone because the group voted to order lunch from a restaurant that wasn't your top pick, how good are you going to be, really, at integrating insights and objections from researchers who disagree with you when it comes to your work, which you hold more precious than a lunch order and with which you are, therefore, probably more susceptible to defensiveness?
For what it's worth, I think people should be cautious before conceding that Nate is a brilliant researcher who does important work.
For a long time, I've been skeptical of MIRI's research program for philosophical/scientific reasons. As I understand it, they started with an orientation around symbolic AI, also known as good old fashioned AI or GOFAI, for short. They had this idea that AI would be a white box, like conventional software, and relatively simple compared to the human brain.
I always thought this was a dubious proposition. I was always partial to biomorphic or neuromorphic approaches to AI, specifically "connectionist" or neural network-based ones. I didn't see any basis for the idea that we could distill intelligence into a manageable number of lines of human-readable code that could be understood by a realistic number of researchers and engineers, like Windows. I didn't see where this idea was coming from and it didn't feel plausible to me.
I seem to recall that as late as maybe 2016, Eliezer Yudkowsky was expressing skepticism toward deep learning as a path to artificial general intelligence (AGI) and still promoting symbolic AI as a more likely alternative. This was years after big results like AlexNet in image recognition and DeepMind's Atari-playing deep reinforcement learning agent, which quickly caused the computer science field to pivot toward deep learning-based approaches to AI. I'm not sure at one point Eliezer made the mental pivot to deep learning as a viable path to AGI. I don't know if he's ever clearly, publicly explained how, when, or why he changed his mind about that. (I find that strange and suspicious. I don't think he likes ever admitting he was wrong about anything.)
I don't know if MIRI has pivoted its approach to AI safety research or AI alignment research from an orientation around symbolic AI to an orientation around deep learning. I don't know if such a thing is even possible. A long time ago, I recall that they hired an intern or junior researcher to work on research related to machine learning, but that person quit after less than a year because they didn't feel they were making enough progress.
MIRI's public discussions of AI alignment are troublingly obscure, almost to the point of mysticism. Eliezer, in particular, seems actively hostile to the idea that other people could ever understand his work and seems to feel it's not his responsibility to make his ideas clear to others. His thinking seems to go: if someone were smart enough to understand his ideas and build on them, they would have independently come up with those ideas already and wouldn't need him in the first place.
So, not only has MIRI historically chosen the wrong paradigm for AI research (symbolic AI vs. deep learning), the researchers closest to MIRI who want to support it even have a hard time understanding its ideas. On the same post I linked to above, I saw this comment from a person whose bio says they are "a research scientist at Google DeepMind on the Scalable Alignment team." They write:
I, personally, have been on the receiving end of (what felt to me like) a Nate-bulldozing, which killed my excitement for engaging with the MIRI-sphere, and also punctured my excitement for doing alignment theory. (Relatedly, Eliezer doing the "state non-obvious takes in an obvious tone, and decline to elaborate" thing which Thomas mentioned earlier.)
It sounds like Nate and Eliezer's ideas are not well-understood even by researchers who are the best-positioned people in the world to understand them. And it sounds like these researchers are eager to understand their ideas, but that at least Eliezer (and probably Nate as well) are unwilling or unable to explain or clarify their ideas.
This makes me ask: why say they do brilliant research if no one can verify it?
I think you're making a lot of assumptions about the community that are at the very least not very obvious. There are some true weirdos around that group (I always think of Aella for some reason,...
I think you're making a lot of assumptions about the community that are at the very least not very obvious. There are some true weirdos around that group (I always think of Aella for some reason, though I don't think she's dangerous in any way) and I think it would make more sense to say that if one suffers from mental illness and is a rationalist, there's a good chance that they're going to be really, really weird, like Ziz here.
But firstly many people on lesswrong are just normal (smart) people interested in the topics discussed, currently for example scientists studying AI safety/alignment who I don't think tend to fall into any of the stereotypes (real or imagined). And secondly I don't get the impression that Ziz was exactly a pillar of the community, even from the article it seemed like they had weird expectations about the community that weren't fullfilled because people in the community aren't unhinged, so Ziz quickly turned against the rationalists.
We could talk about why someone like Ziz even received any space on the forums, but when you make a platform about free discussion, kicking somebody out is probably not an easy decision.
In an abusive relationship, a person who is abusing their partner will often try to isolate them from their friends and family. "Rationalists" promote the idea that people who don't think like them and believe different from them are less rational, less intelligent, less capable, less competent, and not to be trusted. They promote a feeling of alienation from mainstream society.
Based on my experience with reading lesswrong and some individual blogs I don't think this is true at all btw. It seems to be true for Ziz, but again, Ziz didn't seem to be exactly welcome and unchallenged.
See i think this is the trap, and I've seen it with so many smart people. Just because you're smart in one way, does not mean you're superior in all ways, and that's a behavior seen throughout...
are just normal (smart) people
See i think this is the trap, and I've seen it with so many smart people.
Just because you're smart in one way, does not mean you're superior in all ways, and that's a behavior seen throughout time. Also exacerbated by modern society in many ways that says "oh you got A's and all these degrees and gosh you're just so wonderful for it", and never reality checks some of these people with "ok so can you change your oil, or how about handle a lose lose situation, or something wackier like economics that doesn't always behave properly".
"Rationalist" groups seem to attract those with this superiority complex, who are so often unable to be introspective about their possible misunderstandings or weaknesses. Worse they think they're immune to stupidity and wind up on some really dumb arguments because they won't admit their own biases. I think it was slate star codex that has this ranty "vegas shouldn't exist" post that really misses a lot of actual facts to instead ramble about a city they clearly know little about, or the water issues surrounding it and everyone else taking for the colorado river.
One of many smaller examples, and that's before you get into the more egregious out there stuff like "well obviously only smart people should be able to reproduce, so we'll just get a panel of smart people to check if you're smart enough" nonsense you start seeing.
This is a truism and it obviously happens, but I think people who are not very smart in one way overestimate how often it actually happens in reality. My own experience from academia brought some...
Just because you're smart in one way, does not mean you're superior in all ways, and that's a behavior seen throughout time. Also exacerbated by modern society in many ways that says "oh you got A's and all these degrees and gosh you're just so wonderful for it", and never reality checks some of these people with "ok so can you change your oil, or how about handle a lose lose situation, or something wackier like economics that doesn't always behave properly".
This is a truism and it obviously happens, but I think people who are not very smart in one way overestimate how often it actually happens in reality. My own experience from academia brought some frustration from how "generalized brilliant" many of the professors are, including social skills and empathy, and that was in computer science, which is not exactly known for that.
I encounter this much more often either from successful entrepreneurs (mentioned in the recent threads about Vávra from Warhorse, makers of Kingdom Come, and Španěl from Bohemia Interactive, makers of Arma) or from moderately successful engineers who only ever worked in their field of engineering than from academic types.
I think it was slate star codex that has this ranty "vegas shouldn't exist" post that really misses a lot of actual facts to instead ramble about a city they clearly know little about, or the water issues surrounding it and everyone else taking for the colorado river.
I'm too lazy to go check on that article, but here's another thing the author or Slate Star Codex does: he sometimes publishes critical reactions to his controversial articles by other bloggers and responds to them, and he also often publishes a selection of most interesting/important comments below some of his articles, including critical ones. I think there's quite a lot of this in the community.
Sure, and from the little I ever saw, ssc seemed to be more of what they claimed to try to be than a identity movement. Still I've also found some responses (not necessarily from ssc) to be more...
I'm too lazy to go check on that article, but here's another thing the author or Slate Star Codex does: he sometimes publishes critical reactions to his controversial articles by other bloggers and responds to them, and he also often publishes a selection of most interesting/important comments below some of his articles, including critical ones. I think there's quite a lot of this in the community.
Sure, and from the little I ever saw, ssc seemed to be more of what they claimed to try to be than a identity movement. Still I've also found some responses (not necessarily from ssc) to be more "yeah huh i was wrong" and more "well i might have been wrong about that but instead..." because it still stems from a desire to enforce superiority and an inability to accept that they might not be right than any real "rational" motive.
An interesting example from my personal experiences was someone who felt that most of the stock laws were unneeded. If people get scammed they'll just learn not to next time, and it's not like the same scammer could keep doing the same scam over and over, people would learn to not trust them.
There's already an area of discussion there about if that's way to darwinian (not everyone can afford to be scammed more than once), but I feel many of those points, even at the time there were said, felt silly given the actual history of many prominent scammers, let alone entities like musk/trump.
But how do you differ this from a situation where either the author is mostly right or where the point of discussion is a matter of opinion, so it's reasonable and expected that the best outcome...
Still I've also found some responses (not necessarily from ssc) to be more "yeah huh i was wrong" and more "well i might have been wrong about that but instead..." because it still stems from a desire to enforce superiority and an inability to accept that they might not be right than any real "rational" motive.
But how do you differ this from a situation where either the author is mostly right or where the point of discussion is a matter of opinion, so it's reasonable and expected that the best outcome is to agree to disagree? This I think can happen often because, going back to SSC, a lot of the writing is about politics, society and ideology, where differences in opinion are inevitable and frequent. Perhaps you're not suggesting this, but when someone writes a whole article reasoning about how they came to their political or ideological position, I wouldn't expect a couple comments to be good enough for them to go "actually you're right and this article was pointless, oh well."
It may seem that I'm more skeptical about this than warranted, but in the past I've seen too many times people who were disgusted by Slate Star Codex or some others from the community and called them various names for seemingly objective and justified reasons only to find out, after some discussion, that the reason for their hate wasn't really that but "only" a disagreement about their ideology, which was subjective and couldn't really be reasoned about. That was on reddit, and I'm not making assumptions that people are going to be the same here (I wouldn't be here if I did), but it happens.
An interesting example from my personal experiences was someone who felt that most of the stock laws were unneeded. If people get scammed they'll just learn not to next time, and it's not like the same scammer could keep doing the same scam over and over, people would learn to not trust them.
This doesn't exactly surprise me, but the value I see in places like lesswrong is that I would also expect someone to come up and explain in a much more well-reasoned way something like "when you let stupid people get fucked over, they're going to be more and more unhappy and unhappy, stupid, poor people tend to actively work against the goals of a healthy society that you want to live in" - which might sound unemphatetic and borderline antisocial, but it's something understandable for someone who obviously doesn't come from a place of protecting the vulnerable - instead of downvoting and shouting at them for being a disgusting and dumb ancap, as it typically happens on reddit.
I think this approach firstly has an actual chance of changing the person's opinion, but also discussions like that are usually written for other people, maybe more than for the two people arguing, and a civil unemotional approach like that has a much higher chance of convinving someone else whose beliefs are not set in stone.
Well a good starting point is if they are willing to accept people pointing out outright factual innacuracies. If someone starts by saying "the earth if flat and they're hiding it and you can tell...
But how do you differ this from a situation where either the author is mostly right or where the point of discussion is a matter of opinion, so it's reasonable and expected that the best outcome is to agree to disagree?
Well a good starting point is if they are willing to accept people pointing out outright factual innacuracies. If someone starts by saying "the earth if flat and they're hiding it and you can tell because no flights head towards the south/below some spot" (roughly something i've seen before) and they ignore when you argue otherwise, you know this is a sermon not a discussion.
This I think can happen often because, going back to SSC, a lot of the writing is about politics, society and ideology, where differences in opinion are inevitable and frequent. Perhaps you're not suggesting this, but when someone writes a whole article reasoning about how they came to their political or ideological position, I wouldn't expect a couple comments to be good enough for them to go "actually you're right and this article was pointless, oh well."
Sure, but like many who criticize vegas in particular, SSC fell into a some common traps. It's an extra popular topic among "rationalists" because many of them are in the coder/software elite spheres, and thus also often part of FAANG and thus bay area, an area that really hates being told "you should use less water".
A common, and weak, argument is "well if vegas didn't exist...i mean that's stupid right...a city in the desert", when a cursory overview of the facts around the issue quickly reveals that if you erased nevada LA would still be told "don't use so much water" because it's an insanely small % of the allocation.
This is NOT what ssc was talking about or did (at least directly to my memory), but it is a topic i've seen many self identified rationalists opine on from the "rational" point of view that clearly a city in the desert must be taking up all the water, and yet never actually bothering to research how much Nevada, or Vegas, uses, or what other major users of the water are (turns out its dumb shit like growing almonds/alfalfa in climates that don't support it, liiiike california/arizona).
It may seem that I'm more skeptical about this than warranted, but in the past I've seen too many times people who were disgusted by Slate Star Codex or some others from the community and called them various names for seemingly objective and justified reasons only to find out, after some discussion, that the reason for their hate wasn't really that but "only" a disagreement about their ideology, which was subjective and couldn't really be reasoned about. That was on reddit, and I'm not making assumptions that people are going to be the same here (I wouldn't be here if I did), but it happens.
Sure. Rationalists and SSC are extra targeted because they dare to...i guess to summarize fancifully.... explore the taboo that "just do good things!" might not work or that there's a less emotional way to look at things. They will often not just agree that "well x is bad and y stops it so y must be good!"....although I do feel they can essentially dress up the same limitation in logic in more clever trappings.
To be clear, it's a very important viewpoint and discussion to be had. My issue with many rationalists is the same I have with just about every ist/ism/liberal/democrat/republican/conservative/whatever. Namely that often they're unwilling to acknowledge that the system is more complex than their ideology. Rationalists especially are often trying to "solve" things as if there's a single variable that can be plugged in to fix everything forever. I don't identify as anything in these spheres because I've yet to find any group I would say I agree even 80% with. There's a lot of edge cases and what not and by strongly diving into these things it's usually a red flag to me that i'm dealing with someone who's not a fan of nuance.
I will say that I do think that in personal experience I can, on average, at least have a discussion with rationalists, which I think is healthy. Many other such identities do take the "well clearly you just don't understand" stance and shut any conversation down.
This doesn't exactly surprise me, but the value I see in places like lesswrong is that I would also expect someone to come up and explain in a much more well-reasoned way something like "when you let stupid people get fucked over, they're going to be more and more unhappy and unhappy, stupid, poor people tend to actively work against the goals of a healthy society that you want to live in" - which might sound unemphatetic and borderline antisocial, but it's something understandable for someone who obviously doesn't come from a place of protecting the vulnerable - instead of downvoting and shouting at them for being a disgusting and dumb ancap, as it typically happens on reddit.
I think this approach firstly has an actual chance of changing the person's opinion, but also discussions like that are usually written for other people, maybe more than for the two people arguing, and a civil unemotional approach like that has a much higher chance of convinving someone else whose beliefs are not set in stone.
Oh sure I generally agree. I fully acknowledge that what you're talking about happens(both positive discussion and needless attacks), but do want to point out my issue with rationalist groups is different from the "how dare you arrive at capitalism is good" stuff and more that rationalists seem to think they, and their community, is immune to reductive or short sighted thinking because they'll have a rational discussion and arrive at "the answer".
It becomes self fulfilling in many cases and extra dismissive. ESPECIALLY the common "an idiot argued this, and we proved them wrong, so that argument is done" issue. To be clear, I essentially was arguing your example, and it did little to sway them because "they couldn't see that happening"....which is kinda the core of my issue with rationalists. Doubly so when in cases like this, you have historical evidence of it happening over and over, and just an unwillingness to do the legwork to really study it, and instead an assumption of understanding/superiority due to their other talents.
Hey, sorry for seemingly ignoring this comment - I realized I was going to need more time and effort than available to properly reply to it, so I postponed it indefinitely. I want to at least...
Hey, sorry for seemingly ignoring this comment - I realized I was going to need more time and effort than available to properly reply to it, so I postponed it indefinitely. I want to at least acknowledge it to not be rude.
Regarding the straight up nonsense said on lesswrong and SSC, it's possible I extrapolated too much from my experience with the community, which is somewhat limited because it comes only from reading a few interesting people (whether on lesswrong or different platforms), interesting topics and topics shared elsewhere (like here) as controversial or offensive (for which my thershold is apparently higher than average, so those don't exactly disturb me either). So I'm going to adjust my expectations of the community accordingly.
My issue with many rationalists is the same I have with just about every ist/ism/liberal/democrat/republican/conservative/whatever. Namely that often they're unwilling to acknowledge that the system is more complex than their ideology. Rationalists especially are often trying to "solve" things as if there's a single variable that can be plugged in to fix everything forever. I don't identify as anything in these spheres because I've yet to find any group I would say I agree even 80% with.
Right, no disagreement from me here. The reason why I really like some rationalists is not because I 100% agree with them or because I want everyone else to be like them, but because I think that the interent would be a much more bearable place if most people shifted somewhat in the direction of how rationalists think and write about stuff. There are many things that I hate about places like reddit, twitter and other social media, so reading people who sometimes do the opposite of the stuff that pisses me off is incredibly refreshing, that's honestly half of the appeal for me. The fact that sometimes they can be really off as well is not a huge issue for me because I don't think they're any worse than most of the internet with the exception of some high quality traditional media.
I was actually not thinking mainly about any of the people involved in these recent killings, but about a whole bunch of different things involving the Bay Area “rationalist” community, including...
I was actually not thinking mainly about any of the people involved in these recent killings, but about a whole bunch of different things involving the Bay Area “rationalist” community, including bizarre, scary social dynamics at Leverage Research and the Center for Applied Rationality (CFAR). This is a quote from Chapter 2 of “Extropia’s Children”, which is what the parent comment to mine was about and which is what I was specifically replying to:
And then let's note that, summarizing, even according to Alexander (who Taylor disputes in her second post), the Bay Area rationalist community of circa 2017-19 — estimated at “conservatively 500 people,” not exactly a massive horde! — hosted multiple clusters of people who experienced sharp breaks from consensus reality; atleastfour suicides; and at least three firsthand reports, all from clearly brilliant young women, implying / claiming cult-like activity / exploitation.
As Auric Goldfinger might say: “Mr. Bond, they have a saying in Eliezer's hometown. Once is happenstance. Twice is coincidence. The third time, it's enemy action.”
In my experience, even the “rationalists” who seem relatively nice and sensible still say completely crazy stuff sometimes. I remember reading one say that gay people in the 1950s and before must have been evil because being in the closet meant lying and lying is immoral and immoral action implies an overall bad moral character. What??!!
It kind of feels like a hop, skip, and a jump from that level of irrationality to believing that the two hemispheres of your brain are at war with each other and one is trying to kill the other. (Who are you gonna trust, those normie neuroscientists or your own sublime rationality?)
I don’t know if I can properly diagnose or explain the source of these kinds of ridiculous statements. Maybe it has something to do with rejecting academia, journalism, and “consensus reality” and thinking you have cracked the code of rational thought and can rebuild all human knowledge from the ground up from first principles. Which wrongly discounts how much the world relies on the division of cognitive labour and how much cumulative cultural evolution has got to us where we are.
This and many other instances like it made me go from someone who was at least curious about the “rationalist” subculture and willing to hear it out to someone who sees “rationalism” as something the world (and “rationalists”) would be better off without.
I wish this community had called itself something else because by attempting to take ownership of the concept of rationality in its branding (so arrogant!) it has muddied the word “rational”.
I agree this is ridiculous, but I have my doubts about that being said by someone otherwise considered nice and sensible. A day or two ago there was a discussion on Tildes about the supposed...
I remember reading one say that gay people in the 1950s and before must have been evil because being in the closet meant lying and lying is immoral and immoral action implies an overall bad moral character. What??!!
I agree this is ridiculous, but I have my doubts about that being said by someone otherwise considered nice and sensible. A day or two ago there was a discussion on Tildes about the supposed evilness of Scott Alexander for example, surely one of the more controversial people, and based on what I read about him I don't think he would shy away from showing how ridiculous this is..
It kind of feels like a hop, skip, and a jump from that level of irrationality to believing that the two hemispheres of your brain are at war with each other and one is trying to kill the other.
I think it's a bit farther than that, but I can't say I disagree in general.
Maybe it has something to do with rejecting academia, journalism, and “consensus reality”
This is where you lost me because I don't think this is something that commonly happens over lesswrong. For example, the highest rated article at lesswrong a few days (or maybe a week?) ago was by Jan Kulveit, a scientist currently working at Charles University in Prague, in the past at Oxford, who also often speaks in traditional media (most often public radio iirc, so literally as far away from "alternative" media as you can get) around here simply because he's known and respected.
I wish this community had called itself something else because by attempting to take ownership of the concept of rationality in its branding (so arrogant!) it has muddied the word “rational”.
I agree, but also I'm not sure I've ever seen for example the two people I mentioned above use it to describe themselves, other than Scott Alexander commenting on articles about him that used the label.
I suspect that annoying people who meet in groups in California to feel superior and have articles written about them because they're genuinely crazy are a real subgroup distinct from people like Kulveit, but from where I stand they seem like an obvious minority.
It was one of the site admins. Someone who works at Lightcone Infrastructure, the organization that runs the LessWrong forum and that manages a physical meeting space for the "rationalist"...
I agree this is ridiculous, but I have my doubts about that being said by someone otherwise considered nice and sensible.
It was one of the site admins. Someone who works at Lightcone Infrastructure, the organization that runs the LessWrong forum and that manages a physical meeting space for the "rationalist" communist in Berkely.
A day or two ago there was a discussion on Tildes about the supposed evilness of Scott Alexander for example, surely one of the more controversial people, and based on what I read about him I don't think he would shy away from showing how ridiculous this is..
He might publicly say it's ridiculous and then privately say far-right crackpots who believe gay people were evil until after Stonewall might be onto something... No way to tell what he really believes.
This is where you lost me because I don't think this is something that commonly happens over lesswrong.
How familiar are you with LessWrong? A lot of these people are very anti-academia, anti-journalism, anti-"normal".
Not everyone on there is the same and not everyone agrees. But this is a strong trend, including among the more prominent members of the community.
If I recall correctly, there a few people who are somewhat well-known in other circles like AI safety or effective altruism who have posted just a few times total on the forum because they know the community is interested in that topic. These people may not be bought into the community or subculture and may not self-identify as "rationalists".
I also remember reading a post by a person who was once active on the forum and who seems well-known, but said things like (paraphrasing) that posting on LessWrong drove them crazy and the idea of doing it again made them sick. I assume there are others who get to that point and burn out silently. Some people might be attracted to some aspects of the community and then find other aspects of it intolerable.
So, definitely not everyone who has ever participated there can be painted with the same brush. You can probably find at least a few counterexamples to any trend or pattern that holds for most of the community. But then you can also find a hundred examples of the trend.
I agree, but also I'm not sure I've ever seen for example the two people I mentioned above use it to describe themselves, other than Scott Alexander commenting on articles about him that used the label.
I'm confused by this. Are you saying don't think people on LessWrong refer to themselves as "rationalists"? They definitely do, all the time. (There is even a shortened form: "rats".)
They don't use this label with humility, either. I remember reading comments from one or two people saying, more or less, that if police detectives were rationalists and read LessWrong, they would be a lot better at solving murders. I didn't see anyone reply to disagree with or challenge this idea. (And it didn't seem that out of ordinary for how people on the site talk about "rationalism", either.)
Edit: This doesn't directly touch on anything we've talked about in the last few comments, but I added this comment above. This is an example of how, IMO, the "rationalist" community is psychologically unhealthy, not just at the fringes, but at the very centre.
Hey, sorry for not replying. I realized it would require time and effort to reply properly to all of it that I did not have at the time so I postponed it but now I think I better not be rude and...
Hey, sorry for not replying. I realized it would require time and effort to reply properly to all of it that I did not have at the time so I postponed it but now I think I better not be rude and at least acknowledge it.
Regarding your first and third points, the sanity of average lesswrong users: you may well be right. I only read topics that interest me and a few people who interst me (like Kulveit and Scott Alexander), plus here and there topics mentioned as controversial or offensive, and I probably extrapolated too much out of that. I'm adjusting my expectations.
He might publicly say it's ridiculous and then privately say far-right crackpots who believe gay people were evil until after Stonewall might be onto something... No way to tell what he really believes.
I can't open those images, but if they are what I think, I have read those leaked emails and I have no problems with them. To me the emails boil down to "I tried hard to study the available data that seems reliable and based on that I'm worried that IQ differences based on ethnicity might be real, which would likely be very bad for society, so I'm not saying that publicly", which I realize some people have a problem with but I don't, if anything it seems really weird to share this as some shocking discovery. I also don't think this is inconsistent with his public posting, he posted about the same topic very recently, I think in the last two months or so.
Regarding the usage of the term "rationalists", I mostly see it as a bit of tongue in cheek, including the term "rats", but as I mention in the second paragraph, maybe I'm just extrapolating too much from the small subsections of lesswrong that I read. The rationalist interior decorating guide seems like a nice example, I have read that one and the vibe I get from that is self-aware nerdiness - not as in "we're experts" but as in "we know we're not the best at this, so we share among us what seems to work". The links in the section on bright lighting were also very useful for me.
They don't use this label with humility, either. I remember reading comments from one or two people saying, more or less, that if police detectives were rationalists and read LessWrong, they would be a lot better at solving murders.
From what I know about police work, I don't think this is entirely nonsensical, lol. But I get that it's overconfident to the point of being funny.
I still don't see the rationalist label as terrible, but I can't really argue with you regarding the name and rotten core, I have to empathize as I get similarly iritated for probably exactly the same reasions by RationalWiki.
The way the archive site tries to save images from tweets, to see the images you have to right click them and select "Open image in new tab". I don't think it's out of concern for society, it's...
I can't open those images
The way the archive site tries to save images from tweets, to see the images you have to right click them and select "Open image in new tab".
To me the emails boil down to "I tried hard to study the available data that seems reliable and based on that I'm worried that IQ differences based on ethnicity might be real, which would likely be very bad for society, so I'm not saying that publicly"
I don't think it's out of concern for society, it's mainly out of concern for himself. He seems sympathetic to the view that, for example, Black people and women experience worse outcomes not because of racism or sexism, but because of inherent, immutable biological inferiority, and that the system is meritocratic and working as it should. For example, in the email, he says he's annoyed by the amount of time people in tech and computer science have to apologize for being disproportionately white and male.
I don't think he is worried about society being less concerned about racism based on neo-reactionary discourse about race and IQ. He's spent a lot of time and energy over the years trying to convince people to be less concerned about racism. So, that would help him achieve his goal. For example, if more people believed that white people are inherently smarter than Black people, that would reduce the annoying pressure for white people in tech and computer science to apologize that there aren't more Black people in tech and CS.
But he correctly feared that public knowledge of his sympathy toward neo-reactionaries and scientific racism would hurt his reputation, which it has.
Regarding the usage of the term "rationalists", I mostly see it as a bit of tongue in cheek
Alex Jones is not a respected scientist and wasn't called to speak on the radio on, among other things, modelling covid impact and promoting mandatory mask policies. Also by public radio I...
Alex Jones is not a respected scientist and wasn't called to speak on the radio on, among other things, modelling covid impact and promoting mandatory mask policies. Also by public radio I actually meant "publicly funded", in other words Czech Broadcast, which is possibly the highest quality traditional news style medium that we have.
This and deep's sibling comment had me thinking of the information basilisk (see BLIT) and how, in some ways, the Zizians and Vassarites seem to have produced, or at least consumed, one - and...
This and deep's sibling comment had me thinking of the information basilisk (see BLIT) and how, in some ways, the Zizians and Vassarites seem to have produced, or at least consumed, one - and possibly too many psychedelics.
I haven't, and won't be, reading all that, what's the TLDR?
Someone was worried about technological risks and survival of the human species, felt they were intelligent and inspired others into magical thinking like that of Scientologists, Rasputin, and others with secret coveted, dangerous knowledge to develop feelings of being the one true savior (or one of a few). Instead it was "Oops, cult".
It only gets weirder but if you're using psychedelics and you go from "we're all in this together and we should do our part" to "my team and I are only ones who can save us", stop at least the drugs, and maybe the career field you're in.
Is MIRI still selling the narrative of the AI apocalypse, how they're the only ones who can stop it, and therefore you should give them lots and lots of money? I can't tell if it's pure grift or...
Is MIRI still selling the narrative of the AI apocalypse, how they're the only ones who can stop it, and therefore you should give them lots and lots of money? I can't tell if it's pure grift or genuine belief, but it's manipulative in the same way Scientology is. I've heard of people literally tithing to them like a church.
I used to read some of the rationalist forums and blogs out of curiosity, but I've always had a distate for the combination of arrogance and silliness. They have a weird obsession with Bayesian statistics, but don't seem to realize it's a BS in BS out kind of idea, and operate with priors that can't be reasonably estimated as if, just because you ran your completely unsupported guesstimation through a formula, that makes it rational. They freak out over a hypothetical future demon AI. It's all very bizarre. If you have to repeatedly, and seriously, state that you are not a cult, maybe it's time for some self reflection.
There's a long way between "donating money to an AI research organization due to emotional manipulation" and straight up murder, of course. But I'm not surprised that some more unhinged individuals would be attracted to that kind of scene. It appeals to outsiderness and gives people a sense of intellectual and moral superiority, because anything that violates their beliefs isn't just wrong, it's wrong in a way that reflects deeply negatively on the individual.
I heard about this case from a family member and it sadly rings relatable. I was in circles with at least one individual for a fairly long period of time who sounds exactly like the kind of person who would unabashedly support this.
It's odd just how these similar behaviors/fields/beliefs overlap into this kind of group, but it's not a total anomaly other than actually acting on the threats of violence.
Related AP article:
A timeline of activities of a cultlike group tied to the killing of a Border Patrol agent
Edit: Longer articles with more details and an interactive timeline from SF Chronicle:
Who is ‘Ziz’? How a mysterious group with roots in the Bay Area is linked to six deaths
‘Rationalist Fleet’: Before killings linked to fringe group, ‘Ziz’ led fateful tugboat voyage to California
Maps and timeline clarify confusing ‘Ziz’ case linked to deaths of Bay Area landlord, five others
This is related and I found it very interesting: https://aiascendant.substack.com/p/extropias-children
Chapter 2 covers Ziz specifically, but a broader picture of rationalism is probably helpful here
This is a great resource. Thank you.
It makes sense to me that the worldview or subculture that gets called "rationalism" by some Bay Area tech people (not to be confused with the older, more technical, and more specific point of view called "rationalism" from academic philosophy) would be correlated with psychosis. The way people think and write and speak and interact in that world seems almost like it's engineered to make you go insane.
For some reason, my mind is connecting the way that "rationalists" talk to the way that people who are emotionally abusive in intimate relationships talk. I don't know if I can really articulate the connection. I think there is some similarity in terms of talking to you in a way that undermines your trust in yourself — in your ability to reason, your perception, your intelligence, your sanity — and that undermines your trust in and connection to other people.
In an abusive relationship, a person who is abusing their partner will often try to isolate them from their friends and family. "Rationalists" promote the idea that people who don't think like them and believe different from them are less rational, less intelligent, less capable, less competent, and not to be trusted. They promote a feeling of alienation from mainstream society. I can see how that would make a person feel less and less grounded over time the more they are exposed to "rationalist" thought and the more they are immersed in the culture. I can see how, especially if they have other risk factors, it could make a person more susceptible to psychosis.
Now that I'm thinking about it, there's also a more one-to-one correlation, in that many "rationalists" do use bullying tactics and are simply just directly verbally abusive to outsiders and each other. That isn't so unusual for large social platforms like Reddit and Twitter (not that being usual makes it good, safe, or healthy), but it feels more sinister for it to be so common and so normalized in a smaller and more personal online community. (It's also modelled by "leaders"/prominent people in the community).
(By the way, I'm putting "rationalism" and "rationalists" in scare quotes because I just find it so obnoxious and annoying how they see themselves as exemplifying superior rationality and I want to try to give less power to that branding manoeuvre.)
I want to give a citation for my claim that people in this community "use bullying tactics and are simply just directly verbally abusive to outsiders and each other."
I was looking up something completely different and stumbled upon this comment about one of the most prominent members of the Bay Area "rationalist" community, who is the president of the Machine Intelligence Research Institute (MIRI). This comment comes from someone who used to work at MIRI:
What the former employee continues with is kind of sad:
That feels... disturbing?
The former employee tells a few stories about the MIRI president's behaviour, such as:
And:
That's demented.
There are other comments on the same post where people say they've experienced similar behaviour from Nate. A woman who was dating Nate at the time the post was published wrote:
Tellingly, a lot of the comments read to me as variously minimizing, excusing, justifying, or rationalizing this kind of behaviour, using euphemistic language, arguing that people should allowed to behave and communicate in ways that are outside the norm, saying that it must be frustrating for Nate to work with people who aren't as smart as him, etc.
This all lines up with everything else I know about the community norms. Cruelty and emotional violence are largely accepted and not challenged nearly enough.
I worked with Nate quite some time ago now, trying to improve the operations of MIRI. One thing has stayed in my verbal repertoire from my short time working there: "You can't just 'be smart' at it." He had apparently significant domain expertise, but when it came to anything outside of that he was quick, confident, and wrong. Whenever any system was "irrational", it was wrong and it chafed at him to obey the rules. There was no space for just following payroll or administrative law because it was the law, it had to be explained to him until it made sense or until he threw up his hands and gave up fighting it. It was an incredibly exhausting work environment, and I gave up and got away from him very, very quickly.
When I hear stories like this, I can't help but think that a person who putatively has a high level of competence in some technical domain would increase their competence if they got a grip on some of the fundamental human, emotional things, like integrating information and perspectives from other people, the willingness to let go of being right or proving your point, and some kind of balance between or integration of self-doubt and self-confidence. If you're fuming to the point of making other people uneasy or even lashing out at someone because the group voted to order lunch from a restaurant that wasn't your top pick, how good are you going to be, really, at integrating insights and objections from researchers who disagree with you when it comes to your work, which you hold more precious than a lunch order and with which you are, therefore, probably more susceptible to defensiveness?
For what it's worth, I think people should be cautious before conceding that Nate is a brilliant researcher who does important work.
For a long time, I've been skeptical of MIRI's research program for philosophical/scientific reasons. As I understand it, they started with an orientation around symbolic AI, also known as good old fashioned AI or GOFAI, for short. They had this idea that AI would be a white box, like conventional software, and relatively simple compared to the human brain.
I always thought this was a dubious proposition. I was always partial to biomorphic or neuromorphic approaches to AI, specifically "connectionist" or neural network-based ones. I didn't see any basis for the idea that we could distill intelligence into a manageable number of lines of human-readable code that could be understood by a realistic number of researchers and engineers, like Windows. I didn't see where this idea was coming from and it didn't feel plausible to me.
I seem to recall that as late as maybe 2016, Eliezer Yudkowsky was expressing skepticism toward deep learning as a path to artificial general intelligence (AGI) and still promoting symbolic AI as a more likely alternative. This was years after big results like AlexNet in image recognition and DeepMind's Atari-playing deep reinforcement learning agent, which quickly caused the computer science field to pivot toward deep learning-based approaches to AI. I'm not sure at one point Eliezer made the mental pivot to deep learning as a viable path to AGI. I don't know if he's ever clearly, publicly explained how, when, or why he changed his mind about that. (I find that strange and suspicious. I don't think he likes ever admitting he was wrong about anything.)
I don't know if MIRI has pivoted its approach to AI safety research or AI alignment research from an orientation around symbolic AI to an orientation around deep learning. I don't know if such a thing is even possible. A long time ago, I recall that they hired an intern or junior researcher to work on research related to machine learning, but that person quit after less than a year because they didn't feel they were making enough progress.
MIRI's public discussions of AI alignment are troublingly obscure, almost to the point of mysticism. Eliezer, in particular, seems actively hostile to the idea that other people could ever understand his work and seems to feel it's not his responsibility to make his ideas clear to others. His thinking seems to go: if someone were smart enough to understand his ideas and build on them, they would have independently come up with those ideas already and wouldn't need him in the first place.
So, not only has MIRI historically chosen the wrong paradigm for AI research (symbolic AI vs. deep learning), the researchers closest to MIRI who want to support it even have a hard time understanding its ideas. On the same post I linked to above, I saw this comment from a person whose bio says they are "a research scientist at Google DeepMind on the Scalable Alignment team." They write:
It sounds like Nate and Eliezer's ideas are not well-understood even by researchers who are the best-positioned people in the world to understand them. And it sounds like these researchers are eager to understand their ideas, but that at least Eliezer (and probably Nate as well) are unwilling or unable to explain or clarify their ideas.
This makes me ask: why say they do brilliant research if no one can verify it?
I think you're making a lot of assumptions about the community that are at the very least not very obvious. There are some true weirdos around that group (I always think of Aella for some reason, though I don't think she's dangerous in any way) and I think it would make more sense to say that if one suffers from mental illness and is a rationalist, there's a good chance that they're going to be really, really weird, like Ziz here.
But firstly many people on lesswrong are just normal (smart) people interested in the topics discussed, currently for example scientists studying AI safety/alignment who I don't think tend to fall into any of the stereotypes (real or imagined). And secondly I don't get the impression that Ziz was exactly a pillar of the community, even from the article it seemed like they had weird expectations about the community that weren't fullfilled because people in the community aren't unhinged, so Ziz quickly turned against the rationalists.
We could talk about why someone like Ziz even received any space on the forums, but when you make a platform about free discussion, kicking somebody out is probably not an easy decision.
Based on my experience with reading lesswrong and some individual blogs I don't think this is true at all btw. It seems to be true for Ziz, but again, Ziz didn't seem to be exactly welcome and unchallenged.
See i think this is the trap, and I've seen it with so many smart people.
Just because you're smart in one way, does not mean you're superior in all ways, and that's a behavior seen throughout time. Also exacerbated by modern society in many ways that says "oh you got A's and all these degrees and gosh you're just so wonderful for it", and never reality checks some of these people with "ok so can you change your oil, or how about handle a lose lose situation, or something wackier like economics that doesn't always behave properly".
"Rationalist" groups seem to attract those with this superiority complex, who are so often unable to be introspective about their possible misunderstandings or weaknesses. Worse they think they're immune to stupidity and wind up on some really dumb arguments because they won't admit their own biases. I think it was slate star codex that has this ranty "vegas shouldn't exist" post that really misses a lot of actual facts to instead ramble about a city they clearly know little about, or the water issues surrounding it and everyone else taking for the colorado river.
One of many smaller examples, and that's before you get into the more egregious out there stuff like "well obviously only smart people should be able to reproduce, so we'll just get a panel of smart people to check if you're smart enough" nonsense you start seeing.
This is a truism and it obviously happens, but I think people who are not very smart in one way overestimate how often it actually happens in reality. My own experience from academia brought some frustration from how "generalized brilliant" many of the professors are, including social skills and empathy, and that was in computer science, which is not exactly known for that.
I encounter this much more often either from successful entrepreneurs (mentioned in the recent threads about Vávra from Warhorse, makers of Kingdom Come, and Španěl from Bohemia Interactive, makers of Arma) or from moderately successful engineers who only ever worked in their field of engineering than from academic types.
I'm too lazy to go check on that article, but here's another thing the author or Slate Star Codex does: he sometimes publishes critical reactions to his controversial articles by other bloggers and responds to them, and he also often publishes a selection of most interesting/important comments below some of his articles, including critical ones. I think there's quite a lot of this in the community.
Sure, and from the little I ever saw, ssc seemed to be more of what they claimed to try to be than a identity movement. Still I've also found some responses (not necessarily from ssc) to be more "yeah huh i was wrong" and more "well i might have been wrong about that but instead..." because it still stems from a desire to enforce superiority and an inability to accept that they might not be right than any real "rational" motive.
An interesting example from my personal experiences was someone who felt that most of the stock laws were unneeded. If people get scammed they'll just learn not to next time, and it's not like the same scammer could keep doing the same scam over and over, people would learn to not trust them.
There's already an area of discussion there about if that's way to darwinian (not everyone can afford to be scammed more than once), but I feel many of those points, even at the time there were said, felt silly given the actual history of many prominent scammers, let alone entities like musk/trump.
But how do you differ this from a situation where either the author is mostly right or where the point of discussion is a matter of opinion, so it's reasonable and expected that the best outcome is to agree to disagree? This I think can happen often because, going back to SSC, a lot of the writing is about politics, society and ideology, where differences in opinion are inevitable and frequent. Perhaps you're not suggesting this, but when someone writes a whole article reasoning about how they came to their political or ideological position, I wouldn't expect a couple comments to be good enough for them to go "actually you're right and this article was pointless, oh well."
It may seem that I'm more skeptical about this than warranted, but in the past I've seen too many times people who were disgusted by Slate Star Codex or some others from the community and called them various names for seemingly objective and justified reasons only to find out, after some discussion, that the reason for their hate wasn't really that but "only" a disagreement about their ideology, which was subjective and couldn't really be reasoned about. That was on reddit, and I'm not making assumptions that people are going to be the same here (I wouldn't be here if I did), but it happens.
This doesn't exactly surprise me, but the value I see in places like lesswrong is that I would also expect someone to come up and explain in a much more well-reasoned way something like "when you let stupid people get fucked over, they're going to be more and more unhappy and unhappy, stupid, poor people tend to actively work against the goals of a healthy society that you want to live in" - which might sound unemphatetic and borderline antisocial, but it's something understandable for someone who obviously doesn't come from a place of protecting the vulnerable - instead of downvoting and shouting at them for being a disgusting and dumb ancap, as it typically happens on reddit.
I think this approach firstly has an actual chance of changing the person's opinion, but also discussions like that are usually written for other people, maybe more than for the two people arguing, and a civil unemotional approach like that has a much higher chance of convinving someone else whose beliefs are not set in stone.
Well a good starting point is if they are willing to accept people pointing out outright factual innacuracies. If someone starts by saying "the earth if flat and they're hiding it and you can tell because no flights head towards the south/below some spot" (roughly something i've seen before) and they ignore when you argue otherwise, you know this is a sermon not a discussion.
Sure, but like many who criticize vegas in particular, SSC fell into a some common traps. It's an extra popular topic among "rationalists" because many of them are in the coder/software elite spheres, and thus also often part of FAANG and thus bay area, an area that really hates being told "you should use less water".
A common, and weak, argument is "well if vegas didn't exist...i mean that's stupid right...a city in the desert", when a cursory overview of the facts around the issue quickly reveals that if you erased nevada LA would still be told "don't use so much water" because it's an insanely small % of the allocation.
This is NOT what ssc was talking about or did (at least directly to my memory), but it is a topic i've seen many self identified rationalists opine on from the "rational" point of view that clearly a city in the desert must be taking up all the water, and yet never actually bothering to research how much Nevada, or Vegas, uses, or what other major users of the water are (turns out its dumb shit like growing almonds/alfalfa in climates that don't support it, liiiike california/arizona).
Sure. Rationalists and SSC are extra targeted because they dare to...i guess to summarize fancifully.... explore the taboo that "just do good things!" might not work or that there's a less emotional way to look at things. They will often not just agree that "well x is bad and y stops it so y must be good!"....although I do feel they can essentially dress up the same limitation in logic in more clever trappings.
To be clear, it's a very important viewpoint and discussion to be had. My issue with many rationalists is the same I have with just about every ist/ism/liberal/democrat/republican/conservative/whatever. Namely that often they're unwilling to acknowledge that the system is more complex than their ideology. Rationalists especially are often trying to "solve" things as if there's a single variable that can be plugged in to fix everything forever. I don't identify as anything in these spheres because I've yet to find any group I would say I agree even 80% with. There's a lot of edge cases and what not and by strongly diving into these things it's usually a red flag to me that i'm dealing with someone who's not a fan of nuance.
I will say that I do think that in personal experience I can, on average, at least have a discussion with rationalists, which I think is healthy. Many other such identities do take the "well clearly you just don't understand" stance and shut any conversation down.
Oh sure I generally agree. I fully acknowledge that what you're talking about happens(both positive discussion and needless attacks), but do want to point out my issue with rationalist groups is different from the "how dare you arrive at capitalism is good" stuff and more that rationalists seem to think they, and their community, is immune to reductive or short sighted thinking because they'll have a rational discussion and arrive at "the answer".
It becomes self fulfilling in many cases and extra dismissive. ESPECIALLY the common "an idiot argued this, and we proved them wrong, so that argument is done" issue. To be clear, I essentially was arguing your example, and it did little to sway them because "they couldn't see that happening"....which is kinda the core of my issue with rationalists. Doubly so when in cases like this, you have historical evidence of it happening over and over, and just an unwillingness to do the legwork to really study it, and instead an assumption of understanding/superiority due to their other talents.
Hey, sorry for seemingly ignoring this comment - I realized I was going to need more time and effort than available to properly reply to it, so I postponed it indefinitely. I want to at least acknowledge it to not be rude.
Regarding the straight up nonsense said on lesswrong and SSC, it's possible I extrapolated too much from my experience with the community, which is somewhat limited because it comes only from reading a few interesting people (whether on lesswrong or different platforms), interesting topics and topics shared elsewhere (like here) as controversial or offensive (for which my thershold is apparently higher than average, so those don't exactly disturb me either). So I'm going to adjust my expectations of the community accordingly.
Right, no disagreement from me here. The reason why I really like some rationalists is not because I 100% agree with them or because I want everyone else to be like them, but because I think that the interent would be a much more bearable place if most people shifted somewhat in the direction of how rationalists think and write about stuff. There are many things that I hate about places like reddit, twitter and other social media, so reading people who sometimes do the opposite of the stuff that pisses me off is incredibly refreshing, that's honestly half of the appeal for me. The fact that sometimes they can be really off as well is not a huge issue for me because I don't think they're any worse than most of the internet with the exception of some high quality traditional media.
I was actually not thinking mainly about any of the people involved in these recent killings, but about a whole bunch of different things involving the Bay Area “rationalist” community, including bizarre, scary social dynamics at Leverage Research and the Center for Applied Rationality (CFAR). This is a quote from Chapter 2 of “Extropia’s Children”, which is what the parent comment to mine was about and which is what I was specifically replying to:
In my experience, even the “rationalists” who seem relatively nice and sensible still say completely crazy stuff sometimes. I remember reading one say that gay people in the 1950s and before must have been evil because being in the closet meant lying and lying is immoral and immoral action implies an overall bad moral character. What??!!
It kind of feels like a hop, skip, and a jump from that level of irrationality to believing that the two hemispheres of your brain are at war with each other and one is trying to kill the other. (Who are you gonna trust, those normie neuroscientists or your own sublime rationality?)
I don’t know if I can properly diagnose or explain the source of these kinds of ridiculous statements. Maybe it has something to do with rejecting academia, journalism, and “consensus reality” and thinking you have cracked the code of rational thought and can rebuild all human knowledge from the ground up from first principles. Which wrongly discounts how much the world relies on the division of cognitive labour and how much cumulative cultural evolution has got to us where we are.
This and many other instances like it made me go from someone who was at least curious about the “rationalist” subculture and willing to hear it out to someone who sees “rationalism” as something the world (and “rationalists”) would be better off without.
I wish this community had called itself something else because by attempting to take ownership of the concept of rationality in its branding (so arrogant!) it has muddied the word “rational”.
[Edited to add a few sentences.]
I agree this is ridiculous, but I have my doubts about that being said by someone otherwise considered nice and sensible. A day or two ago there was a discussion on Tildes about the supposed evilness of Scott Alexander for example, surely one of the more controversial people, and based on what I read about him I don't think he would shy away from showing how ridiculous this is..
I think it's a bit farther than that, but I can't say I disagree in general.
This is where you lost me because I don't think this is something that commonly happens over lesswrong. For example, the highest rated article at lesswrong a few days (or maybe a week?) ago was by Jan Kulveit, a scientist currently working at Charles University in Prague, in the past at Oxford, who also often speaks in traditional media (most often public radio iirc, so literally as far away from "alternative" media as you can get) around here simply because he's known and respected.
I agree, but also I'm not sure I've ever seen for example the two people I mentioned above use it to describe themselves, other than Scott Alexander commenting on articles about him that used the label.
I suspect that annoying people who meet in groups in California to feel superior and have articles written about them because they're genuinely crazy are a real subgroup distinct from people like Kulveit, but from where I stand they seem like an obvious minority.
It was one of the site admins. Someone who works at Lightcone Infrastructure, the organization that runs the LessWrong forum and that manages a physical meeting space for the "rationalist" communist in Berkely.
He might publicly say it's ridiculous and then privately say far-right crackpots who believe gay people were evil until after Stonewall might be onto something... No way to tell what he really believes.
How familiar are you with LessWrong? A lot of these people are very anti-academia, anti-journalism, anti-"normal".
Not everyone on there is the same and not everyone agrees. But this is a strong trend, including among the more prominent members of the community.
If I recall correctly, there a few people who are somewhat well-known in other circles like AI safety or effective altruism who have posted just a few times total on the forum because they know the community is interested in that topic. These people may not be bought into the community or subculture and may not self-identify as "rationalists".
I also remember reading a post by a person who was once active on the forum and who seems well-known, but said things like (paraphrasing) that posting on LessWrong drove them crazy and the idea of doing it again made them sick. I assume there are others who get to that point and burn out silently. Some people might be attracted to some aspects of the community and then find other aspects of it intolerable.
So, definitely not everyone who has ever participated there can be painted with the same brush. You can probably find at least a few counterexamples to any trend or pattern that holds for most of the community. But then you can also find a hundred examples of the trend.
I'm confused by this. Are you saying don't think people on LessWrong refer to themselves as "rationalists"? They definitely do, all the time. (There is even a shortened form: "rats".)
Example post: "Guide to rationalist interior decorating".
They don't use this label with humility, either. I remember reading comments from one or two people saying, more or less, that if police detectives were rationalists and read LessWrong, they would be a lot better at solving murders. I didn't see anyone reply to disagree with or challenge this idea. (And it didn't seem that out of ordinary for how people on the site talk about "rationalism", either.)
Edit: This doesn't directly touch on anything we've talked about in the last few comments, but I added this comment above. This is an example of how, IMO, the "rationalist" community is psychologically unhealthy, not just at the fringes, but at the very centre.
Hey, sorry for not replying. I realized it would require time and effort to reply properly to all of it that I did not have at the time so I postponed it but now I think I better not be rude and at least acknowledge it.
Regarding your first and third points, the sanity of average lesswrong users: you may well be right. I only read topics that interest me and a few people who interst me (like Kulveit and Scott Alexander), plus here and there topics mentioned as controversial or offensive, and I probably extrapolated too much out of that. I'm adjusting my expectations.
I can't open those images, but if they are what I think, I have read those leaked emails and I have no problems with them. To me the emails boil down to "I tried hard to study the available data that seems reliable and based on that I'm worried that IQ differences based on ethnicity might be real, which would likely be very bad for society, so I'm not saying that publicly", which I realize some people have a problem with but I don't, if anything it seems really weird to share this as some shocking discovery. I also don't think this is inconsistent with his public posting, he posted about the same topic very recently, I think in the last two months or so.
Regarding the usage of the term "rationalists", I mostly see it as a bit of tongue in cheek, including the term "rats", but as I mention in the second paragraph, maybe I'm just extrapolating too much from the small subsections of lesswrong that I read. The rationalist interior decorating guide seems like a nice example, I have read that one and the vibe I get from that is self-aware nerdiness - not as in "we're experts" but as in "we know we're not the best at this, so we share among us what seems to work". The links in the section on bright lighting were also very useful for me.
From what I know about police work, I don't think this is entirely nonsensical, lol. But I get that it's overconfident to the point of being funny.
I still don't see the rationalist label as terrible, but I can't really argue with you regarding the name and rotten core, I have to empathize as I get similarly iritated for probably exactly the same reasions by RationalWiki.
The way the archive site tries to save images from tweets, to see the images you have to right click them and select "Open image in new tab".
I don't think it's out of concern for society, it's mainly out of concern for himself. He seems sympathetic to the view that, for example, Black people and women experience worse outcomes not because of racism or sexism, but because of inherent, immutable biological inferiority, and that the system is meritocratic and working as it should. For example, in the email, he says he's annoyed by the amount of time people in tech and computer science have to apologize for being disproportionately white and male.
I don't think he is worried about society being less concerned about racism based on neo-reactionary discourse about race and IQ. He's spent a lot of time and energy over the years trying to convince people to be less concerned about racism. So, that would help him achieve his goal. For example, if more people believed that white people are inherently smarter than Black people, that would reduce the annoying pressure for white people in tech and computer science to apologize that there aren't more Black people in tech and CS.
But he correctly feared that public knowledge of his sympathy toward neo-reactionaries and scientific racism would hurt his reputation, which it has.
It's absolutely not.
...Alex Jones got his start on public radio.
Alex Jones is not a respected scientist and wasn't called to speak on the radio on, among other things, modelling covid impact and promoting mandatory mask policies. Also by public radio I actually meant "publicly funded", in other words Czech Broadcast, which is possibly the highest quality traditional news style medium that we have.
This and deep's sibling comment had me thinking of the information basilisk (see BLIT) and how, in some ways, the Zizians and Vassarites seem to have produced, or at least consumed, one - and possibly too many psychedelics.
I haven't, and won't be, reading all that, what's the TLDR?
Someone was worried about technological risks and survival of the human species, felt they were intelligent and inspired others into magical thinking like that of Scientologists, Rasputin, and others with secret coveted, dangerous knowledge to develop feelings of being the one true savior (or one of a few). Instead it was "Oops, cult".
It only gets weirder but if you're using psychedelics and you go from "we're all in this together and we should do our part" to "my team and I are only ones who can save us", stop at least the drugs, and maybe the career field you're in.
Is MIRI still selling the narrative of the AI apocalypse, how they're the only ones who can stop it, and therefore you should give them lots and lots of money? I can't tell if it's pure grift or genuine belief, but it's manipulative in the same way Scientology is. I've heard of people literally tithing to them like a church.
I used to read some of the rationalist forums and blogs out of curiosity, but I've always had a distate for the combination of arrogance and silliness. They have a weird obsession with Bayesian statistics, but don't seem to realize it's a BS in BS out kind of idea, and operate with priors that can't be reasonably estimated as if, just because you ran your completely unsupported guesstimation through a formula, that makes it rational. They freak out over a hypothetical future demon AI. It's all very bizarre. If you have to repeatedly, and seriously, state that you are not a cult, maybe it's time for some self reflection.
There's a long way between "donating money to an AI research organization due to emotional manipulation" and straight up murder, of course. But I'm not surprised that some more unhinged individuals would be attracted to that kind of scene. It appeals to outsiderness and gives people a sense of intellectual and moral superiority, because anything that violates their beliefs isn't just wrong, it's wrong in a way that reflects deeply negatively on the individual.