26 votes

‘The Body Keeps the Score’ offers uncertain [and misleading] science in the name of self-help. It’s not alone.

13 comments

  1. [7]
    chocobean
    Link
    Um, this was what was taught to me, a millenial AP psych student, along with Stanford and Milgrim, which I suspect are still uncritically printed on many textbooks today. The common allure of...

    Maybe you’ve heard of it as your “lizard brain” or “reptilian brain” — the idea that humans have a primitive brain center that acts on instinct. The triune brain model, developed in the mid-20th century by neuroscientist Paul MacLean, has by now long been considered nonsense by most neuroscientists.

    Um, this was what was taught to me, a millenial AP psych student, along with Stanford and Milgrim, which I suspect are still uncritically printed on many textbooks today.

    ...But there is no remedy for the human condition, and implying that it’s all in our heads makes our pain seem like a glitch instead of a feature.

    The common allure of pseudoscience, I think isn't highlighted by the article author. We live in a society that fail to acknowledge trauma properly, one that tell neglected adult children there's nothing at all the matter with their parents, one that dismiss pain in others as attention seeking or simple failure to grow up. Having a biological, physical, and hence real cause for the internal, invisible and oft dismissed parts of ourselves be taken seriously provides an outlet for our wishes to be taken seriously by others.

    If my internal hurt is as visible as a missing limb, perhaps others would see my dis-ability as a real condition instead of merely a personal weakness.

    24 votes
    1. [5]
      daywalker
      Link Parent
      It's so wild that Stanford and Milgram experiments are still considered good science in some teaching departments and books. Especially the Stanford experiment was such nonsense. Every time I see...

      Um, this was what was taught to me, a millenial AP psych student, along with Stanford and Milgrim, which I suspect are still uncritically printed on many textbooks today.

      It's so wild that Stanford and Milgram experiments are still considered good science in some teaching departments and books. Especially the Stanford experiment was such nonsense. Every time I see it mentioned in a book as if it's replicated and good science, I lose respect for that book and the authors.

      Semi-related, but while looking for critiques of the book, I found this discussion about the book on a physician student site (that also has older members that are now practitioners). I haven't read the entirety of it yet, but what I read was interesting.

      12 votes
      1. V17
        Link Parent
        There are some interesting points there, but do take them with a grain or salt (ideally check them yourself, but that's not realistic) because IME young doctors and especially students tend to be...

        Semi-related, but while looking for critiques of the book, I found this discussion about the book on a physician student site (that also has older members that are now practitioners). I haven't read the entirety of it yet, but what I read was interesting.

        There are some interesting points there, but do take them with a grain or salt (ideally check them yourself, but that's not realistic) because IME young doctors and especially students tend to be pretty bad about interpreting science, especially that which is unclear and difficult to do. They usually don't understand the intricacies of research that's simply difficult and research that's kind of badly done but still brings interesting pointers despite that (which does happen and it's not even that uncommon), and they're very likely to just throw the baby out with the bathwater on things they don't like.

        I'm not saying the book is good, but what I am saying is that if it does have good points among the unsupported parts (which is often the case in these situations), this is not the place to look for them.

        7 votes
      2. [3]
        BeanBurrito
        Link Parent
        Are you a professional in that field?

        It's so wild that Stanford and Milgram experiments are still considered good science in some teaching departments and book

        Are you a professional in that field?

        3 votes
        1. daywalker
          (edited )
          Link Parent
          No, but I am a scientist and I'm educated on -and interested in- experimental design. These skills, to some degree, transfer between disciplines. Especially if you're trying to learn more about...
          • Exemplary

          No, but I am a scientist and I'm educated on -and interested in- experimental design. These skills, to some degree, transfer between disciplines. Especially if you're trying to learn more about other disciplines and engaging with their literature and methodologies, which I've been trying to do for some years. For example, double-blind experimental design is a key component in this type of experimentation. But let alone doing that, both Stanford and Milgram had experimenters actively interfering with the experiments, in order to influence the outcome to get the results they wanted. You can check the resources I will share below for that.

          There are also plenty of cases of people from the psychology and related fields laying out why they were awful experiments, especially the Stanford experiment. Below are some examples.

          Stanford Prison Experiment

          This relatively recent paper, and the citations there, are a good example: Debunking the Stanford Prison Experiment (2019)

          There is also this paper solely dedicated to cricizing its framing in introductory psychology textbooks: Coverage of the Stanford Prison Experiment in Introductory Psychology Textbooks (2014)

          There is another critical paper that assesses its coverage in introductory sociology textbooks: Coverage of the Stanford Prison Experiment in Introductory Social Psychology Textbooks (2014)

          Here is also a teaching paper about how awful the Stanford Prison Experiment was, and why it should be used as an example to teach students about recognizing bullshit science (paraphrasing): Teaching Scientific Thinking Using Recent Archival Revelations about the Stanford Prison Experiment (2019)

          If I humor myself a little, I'd say that debunking of Stanford prison experiment's findings and methodology is repeated so much that it could be considered a subfield by itself.

          If anyone wants to read a layperson explanation, there's this Vox article on it: The Stanford Prison Experiment was massively influential. We just learned it was a fraud.

          Milgram Obedience Experiment

          There are also plenty of criticisms of the Milgram experiment. Here are some papers (and a chapter), similar in nature, to the Stanford ones.

          There is also this paper that studied the personal notes of Milgram and the published results, and found that he falsified the published data: Deception and Illusion in Milgram’s Accounts of the Obedience Experiments

          There's a key difference between Stanford and Milgram experiments, though, and that is the replication part. Despite these very valid criticisms, Milgram's findings were to some degree significantly replicated in other settings. That's one reason why I said Stanford was worse. With this being said, these follow-up studies don't dismiss the criticisms about the original experiment. Replication was more due to luck than proper experimental design.

          Conclusion

          Both experiments are so shady and awfully designed that for decades people in psychology and related fields have been pushing to get them out of the introductory textbooks, or at the very least include the very valid criticisms. These criticisms are also so frequent that wiki entries of both experiments have sections dedicated to them: here and here. Of course, it's Wiki, so the coverage is not even remotely expansive, but it supports my point that criticims are so frequent that Wiki editors felt the need to add sections dedicated to them.

          You can again check out the Vox article for a layperson explanation, as it covers this experiment at some point.

          Personally, I think both of these experiments are still so popular because they seem edgy and have ideological implications. They are both also very simplistic. However, this is no excuse for teaching bad science, and both of these experiments have bullshit designs and even fraud.

          Edit: You can also look up the term "metascience" to get an idea about where I'm coming from. If you're interested in science of science, you can more easily spot flaws in experimental design in other fields.

          26 votes
        2. RoyalHenOil
          Link Parent
          You don't need to be a professional in a specific field to find flaws in the way a study in that field is conducted or reported. The scientific method is broadly applicable, and psychologists can...

          You don't need to be a professional in a specific field to find flaws in the way a study in that field is conducted or reported. The scientific method is broadly applicable, and psychologists can and should be held to the same standards of rigor as other fields.

          I have never looked into the Stanford experiment (it always struck me as too silly to spend much time thinking about), but I have looked into the Milgram experiment because it seems quite plausible and meaningful at first glance.

          Unfortunately, it lacks rigor. A key component of good experiment design involves controlling or correcting for extraneous factors that can influence the results. But in the Milgram study, experimenters frequently went off script and varied in the level of social pressure they applied to test subjects. It is nonsense to say that 65% of participants were susceptible to obeying orders when the orders themselves were all over the place.

          Unfortunately, this extends beyond just poor science and creeps well into the territory of fraud because Stanley Milgram actively reported the script given to experimenters, but did not report on experimenters diverging from the script. He recorded the conversations between experimenters and subjects, so he knew that a lot of the data was biased, but he chose to include the compromised data in his results anyway.

          Additionally, many of the subjects knew that the electric shocks were fake. They correctly surmised that Yale wasn't going to rustle up random volunteers and then have experimenters just casually ask other volunteers to deliver painful shocks to them while they cried out in agony and begged for relief. Experimenters collected data on skepticism expressed by test subjects after each session, yet Milgram did not see fit to report this or to exclude those subjects' results from his findings.

          17 votes
    2. sparksbet
      Link Parent
      To be fair, as a zillennial, the Stanford prison experiment was primarily taught as a "see what would happen if we didn't require IRB approval?" type thing. And I never even encountered the idea...

      To be fair, as a zillennial, the Stanford prison experiment was primarily taught as a "see what would happen if we didn't require IRB approval?" type thing. And I never even encountered the idea of a reptilian brain in my psych classes. But I also didn't take psych until college, where I suspect things are updated a little faster.

      5 votes
  2. daywalker
    Link
    Archive.is link that bypasses paywall I also added the [and misleading] part in the title myself, because some of the criticisms make that apparent. The discussion about the dorsal vagal complex's...

    Archive.is link that bypasses paywall

    I also added the [and misleading] part in the title myself, because some of the criticisms make that apparent. The discussion about the dorsal vagal complex's absence in humans is a good example. Another one is the use of triune brain model in the book, even though it's considered obsolete.

    8 votes
  3. [5]
    TonesTones
    Link
    I don’t like speaking about my journey about mental health publicly. For reasons you’ll see in this post, I don’t think a lot of what I have to say will actually help anyone. I also haven’t read...

    I don’t like speaking about my journey about mental health publicly. For reasons you’ll see in this post, I don’t think a lot of what I have to say will actually help anyone.

    I also haven’t read any of the literature mentioned in this critique. I’ve had the book highly commended to me by peers, but I don’t consider myself a victim of any kind of major trauma. Therefore, I will refrain from commenting on those books directly.

    That being said, I think this post hits the nail on the head with the issue with society’s commodification of mental health.

    I think trying to apply science to mental health is a largely modern phenomena, and while more effective than no treatment, seems to label bad mental health as a problem to be solved with treatments rather than an element of the human condition.

    It’s appealing, but that doesn’t capture the whole picture for me. As my mental health has improved, the “why” is best explained by Eastern ideas rather than science. I’m a scientist, and I dislike how far Eastern ideas stray from falsifiability. However, I’ve seen my mental health improve by realizing truths in my head that I can’t really even put into language. And my best attempts always form dialectics——the namesake of DBT——which are logically contradictory statements that remain true.

    These ideas are really most explored in Easteen philosophy. For example, Buddhism says that one cannot be taught some truths, but they must be experienced and realized. The monks might be able to show you the path, but you must walk it. I can’t help anyone get “better mental health”, really. I can’t even say, let alone teach, what has helped me.

    I don’t practice Buddhism. There are things I dislike about Eastern philisophy. Maybe someday I’ll change my mind. But for now, it’s by far the best analogous comparison I can make to my journey to improve my mental health. And even then, I don’t even know if my mental health has “improved”. I’m just better at living peacefully and effectively with my conditions.

    I say all this since it makes sense these books, that so many say are incredibly helpful, preach bad science.

    Even O’Connor acknowledges early on that she does “not believe that a neuroscientific perspective on grief is any better than a sociological, a religious, or an anthropological one,” but she feels that “neuroscience is part of the conversation of our times.”

    Again, I haven’t read them, but I’d guess that many of the readers that love these books find value in them not for the science, but because the books show these people the path to acceptance of their situation, and therefore the path to relieve their misery.

    But the neuroscience, the author’s credentials, and everything else seems to be modern marketing. I doubt their experience in neuroscience is what makes these books so widely praised. After all, no good therapist has their client read the most recent neuroscience literature, and that’s not because they can’t understand it. It’s just not helpful.

    8 votes
    1. [4]
      Wolf_359
      (edited )
      Link Parent
      Hi there, I half agree with you! I'm a skeptical person by nature and i tend to be dismissive of unscientific things. Emotions are pretty complex however, and science struggles to capture the...

      Hi there, I half agree with you!

      I'm a skeptical person by nature and i tend to be dismissive of unscientific things. Emotions are pretty complex however, and science struggles to capture the nuances of thought and feeling.

      I think most of what you're describing can be explained and quantified in a scientific way. It's just very reductive and unhelpful for the people who need it. I'll give you an example: A 23-year-old heroin addict realizes he is on a self-destructive path and encounters several beautiful, moving pieces of art that speak to him in that moment of immense suffering. He speaks to people who share nuggets of wisdom, which help him realize there is a better way to move through the world. He is healed and able to better cope with his pain.

      A scientific journal would likely make him a single data point in a study that says, Survey respondents who reported feelings of control, a high ability to practice gratitude, and a "sense of oneness with other people" were 9 times less likely to relapse into substance abuse in the following five year period.

      You read that and it makes sense, but it's not fixing your broken heart. Meanwhile, you read something beautiful or see the right piece of art, and it can tear the rot out of your soul and replace it with love. You go through a bad experience and grow from it in ways you couldn't have imagined.

      I was that 23-year-old, and I could write a novel about my experiences without even scratching the surface. I had to experience it myself. I had to hear the right things at the right time in my life to be moved by them. I had to encounter a life-changing piece of art in exactly the right mindset for it to sink in and change my understanding of the world.

      How do I capture any of that in a survey? How does a researcher draw conclusions that will be meaningful to someone else who will need different things at different times to deal with different problems?

      When we call something an art, we are just saying it's a super complex science we don't quite understand yet. And right now, healing a broken person is an art.

      8 votes
      1. [2]
        TonesTones
        (edited )
        Link Parent
        I think we’re mostly on the same page. Edit: Thanks for taking the time to write such a thorough response. I’m happy for you and your journey to healing. I needed to edit this in; sometimes I get...

        I think we’re mostly on the same page. Edit: Thanks for taking the time to write such a thorough response. I’m happy for you and your journey to healing. I needed to edit this in; sometimes I get caught up in the substance of the text and forget the person behind the screen.

        I actually cut out a decent portion of my original response for brevity, and upon rereading, I notice I now never introduce what DBT is before using the abbreviation.

        A scientific journal would likely make him a single data point in a study […]

        I think the portion of science that takes treatments and analyzes their efficacy across a large sample size is helpful. It provides a basis for therapists’ training and helps society get from mental asylums to helpful holistic treatment centers (we’re still on that path).

        Dialectical Behavioral Therapy (DBT) comes from that field; from the patient’s perspective, it’s essentially a collection of extremely well-defined tools (I know it’s more than that, but I’m not qualified to speak on the therapist’s training portion). The large-dataset studies on it are promising; it works for a lot of people (but not everyone). I do think it’s telling that an effective treatment can be compared to ‘well-defined throwing shit at the wall’. No DBT patient (I think) uses every one of the tools in their day-to-day life. They may try them all, and then use what’s helpful.

        I think there’s a community that try to provide a unified theory for some of our mental ailments, and at the moment, they pull from neuroscience to get there. Neuroscience isn’t well-equipped to do that right now, in my opinion. So pieces like this one from the Washington Post appear essentially calling that unified explanation psuedoscience. Hearing those “this is why” explanations can be nice, even if they are wrong. I think our mental health is too emergent from our past experiences to have a “one size fits all” solution. So I think that messaging largely serves as marketing.

        I suspect that if such a solution ever appears, it will take the form of DBT, with so many different tools that you get complete or near-complete coverage. And even then, I don’t think either the doctor or the patient will truly know the ‘cause’ of what was wrong to begin with. They might get a vague approximation, like most complex issues in science get, but it won’t be complete.

        I removed that nuance in my original comment; it wasn’t the focus of what I was trying to communicate. I do think a scientific approach is helpful from a treatment perspective. I just don’t think it will ever provide a complete explanation as to what the underlying issue is.

        1. Wolf_359
          Link Parent
          I think you nailed it. Basically, it's important to know the science behind mental health in general, but each case will be unique to the individual. I loved your comparison to engineering by the...

          I think you nailed it. Basically, it's important to know the science behind mental health in general, but each case will be unique to the individual. I loved your comparison to engineering by the way. I think the multi-pronged approach is right. Mental health requires science, engineering, and a bit of art to get right.

          I'm going to read more about DBT

          1 vote
      2. TonesTones
        Link Parent
        I decided to make a second reply, since I wanted to speak to something else you wrote. To me, science is the process of developing falsifiable hypotheses about phenomena and testing them. I don’t...

        I decided to make a second reply, since I wanted to speak to something else you wrote.

        When we call something an art, we are just saying it’s a super complex science we don’t quite understand yet.

        To me, science is the process of developing falsifiable hypotheses about phenomena and testing them. I don’t think art can be contained within that definition of science.

        Art feels too complex, from the author’s experience, to the viewer’s past experiences, to ever be contained within the hypothesis-test framework. Each piece is individual.

        And right now, healing a broken person is an art.

        I think I agree here. The ways in which people are broken also feels too individual to be captured within the hypothesis-test framework. You might have a statistical prediction about how effective a treatment will be across a large sample size, but that’s probably the best you can do.

        Improving mental health feels like a mix between engineering and art. To me, engineering captures the iterative process of developing a solution, finding holes, and improving it. (That’s a dramatic simplification for some fields, but does capture non-traditional engineering like Software Engineering.)

        We iterate on treatments, solutions and tools for specific people until they are able to get better. However, I think most people (including myself) have a few moments like you describe in their journey; a few almost-magical intersections of right place, time, mindset, etc. That can’t be iterated towards, it just has to happen, and that seems to me like art.