29 votes

Does history have a replication crisis?

18 comments

  1. [2]
    skybrian
    Link
    From the article: ... ... ... ...

    From the article:

    Historical myths, often based on mere misunderstanding, but occasionally on bias or fraud, spread like wildfire. People just love to share unusual and interesting facts, and history is replete with things that are both unusual and true. So much that is surprising or shocking has happened, that it can take only years or decades of familiarity with a particular niche of history in order to smell a rat. Not only do myths spread rapidly, but they survive — far longer, I suspect, than in scientific fields.

    ...

    Take the oft-repeated idea that more troops were sent to quash the Luddites in 1812 than to fight Napoleon in the Peninsular War in 1808. Utter nonsense, as I set out in 2017, though it has been cited again and again and again as fact ever since Eric Hobsbawm first misled everyone back in 1964. Before me, only a handful of niche military history experts seem to have noticed and were largely ignored. Despite being busted, it continues to spread. Terry Deary (of Horrible Histories fame), to give just one of many recent examples, repeated the myth in a 2020 book. Historical myths are especially zombie-like. Even when disproven, they just. won’t. die.

    Or take the case of the 12,000-franc prize instituted by Napoleon for an improved method of preserving food for the use of his armies, which prompted Nicolas Appert to invent canned food. It’s frequently cited to show the how prizes can have a significant impact. Except that, despite being repeated hundreds of times, it literally never happened. Appert was given money by the French government, but it was a mere reward in recognition of his achievement, given over a decade after he had invented the method. The myth of the food canning innovation prize is a truly ancient one, which I traced back to a mis-translation of a vaguely-worded French source all the way back in 1869. That’s over 150 years of repeated falsehood, and I see no signs of it slowing.

    ...

    If the lack of replication or reproducibility is a problem in science, in history nobody even thinks about it in such terms. I don’t think I’ve ever heard of anyone systematically looking at the same sources as another historian and seeing if they’d reach the same conclusions. Nor can I think of a history paper ever being retracted or corrected, as they can be in science.

    ...

    This lack of effective institutions or incentives was really brought home to me recently by the publication of a paper in the prestigious journal History & Technology by Jenny Bulstrode of UCL, in which she claimed that the inventor Henry Cort had stolen his famous 1783 iron-rolling process from Reeder’s iron mill in Jamaica, where it had been developed by 76 black metallurgists by passing iron through grooved sugar rollers. It was a widely-publicised paper, receiving 22,756 views — eleven times as many views as the journal’s next most most read paper, and frankly unheard of for most academic papers — along with a huge amount of press coverage.

    ...

    After writing it, I was contacted by Oliver Jelf, currently completing a Masters thesis, who had actually bothered to go and check the main sources that Bulstrode cites, and has not only examined her interpretation of the sources in detail, but has transcribed those same sources for anyone to read and judge for themselves. Jelf notes that significant parts of Bulstrode’s story do not follow from the evidence she cites [....]

    16 votes
    1. Trauma
      Link Parent
      It seems I've been naive about history as an academic field, probably because historians are presented in media as an argumentative bunch that constantly feud about their pet theories and wouldn't...

      It seems I've been naive about history as an academic field, probably because historians are presented in media as an argumentative bunch that constantly feud about their pet theories and wouldn't miss an opportunity to invalidate the opinions of their opponents. Well, writing it out does indeed make it sound naive. Other academic fields are much more competitive because they offer money and fame, and even there people rarely bother to check their peers and rather focus on outracing them.

      8 votes
  2. [6]
    Comment deleted by author
    Link
    1. [2]
      R3qn65
      Link Parent
      What she wrote was a perfect fit for our current culture - perfectly poised to "go viral," as it were. To summarize her assertion, white industrialist Henry Cort did not invent a new process, as...
      • Exemplary

      What she wrote was a perfect fit for our current culture - perfectly poised to "go viral," as it were. To summarize her assertion, white industrialist Henry Cort did not invent a new process, as is commonly believed, but instead stole the idea from a group of black workers in Jamaica. (As the article discusses, this is probably not true.)

      What she wrote touches on a lot of contemporary left-leaning flashpoints. Industrialists are bad, white industrialists are worse, and colonial civilizations stole things from black people. Here's a sentence from a website discussing Bullstrode's work:

      It is essential to acknowledge and recognize the true inventors of this metallurgical process, as it highlights the systemic erasure of Black contributions to science and industry. By reclaiming their rightful place in history, we can challenge the dominant narrative and celebrate the achievements of Black scientists and innovators.

      Celebrating the achievements of black innovators is a great thing (hopefully that will soon go without saying.) What I'm more skeptical of is having "challenge the dominant narrative" be a goal in and of itself.

      14 votes
      1. skybrian
        Link Parent
        Yes, in academic circles, this article was doing something more like reinforcing a dominant narrative while “challenging” an older one that was formerly dominant. Or maybe we should just avoid...

        Yes, in academic circles, this article was doing something more like reinforcing a dominant narrative while “challenging” an older one that was formerly dominant.

        Or maybe we should just avoid assumptions about which narratives are dominant, because it depends on which community you mean. There are competing narratives. Figuring out what happened is about understanding evidence and choosing how to tell a story about it should come second.

        3 votes
    2. [3]
      EarlyWords
      Link Parent
      I’ve been running an ancient history YouTube channel for the last few years and it is the most confounding thing in the world trying to figure out what the public-at-large will like. So much is...

      I’ve been running an ancient history YouTube channel for the last few years and it is the most confounding thing in the world trying to figure out what the public-at-large will like. So much is timing and cultural context. It’s as easy to be too early with a new concept as it is to be too late, when it’s already been discussed and decided upon.

      We like to think of earlier millennia as impervious to the vagaries of current-day trends but the lens through which we see everything is far more powerful than we’d care to admit. Our channel follows academia and scholarly consensus so the rise and fall of certain subjects, like economic history or intersectional issues, is quite clear in the current discourse.

      7 votes
      1. [2]
        Toric
        Link Parent
        My only exposure to the ancient history crowd has been overly sarcastic productions, which I originally joined fro Reds mythology videos but have grown to love Blues ancient history videos just as...

        My only exposure to the ancient history crowd has been overly sarcastic productions, which I originally joined fro Reds mythology videos but have grown to love Blues ancient history videos just as mutch. Whats your opinion on them / history communicators on YT in general? Are there times the truth is stretched for a better narriative?

        1. EarlyWords
          Link Parent
          I don’t know that specific channel. But what YouTube makes content creators do is probably akin to the pressures researchers and academics feel from publishers and their own administrations. Very...

          I don’t know that specific channel. But what YouTube makes content creators do is probably akin to the pressures researchers and academics feel from publishers and their own administrations. Very often the requirements are extremely random or reactive to such ephemeral fads that strategy becomes impossible.

          In this environment, producers are desperate to grab eyeballs whatever way they can. Very few channels make any money from being accurate, only contentious. We have to verify every assertion we make. Many of these folks don’t do that.

  3. [11]
    Asinine
    Link
    Reminds me of those who say that religion has one of the highest body counts. All these are based on Wikipedia's stats (this was done in 2014, so I no longer have the original data saved. But it's...

    Reminds me of those who say that religion has one of the highest body counts.

    All these are based on Wikipedia's stats (this was done in 2014, so I no longer have the original data saved. But it's on the wiki.)

    Deaths from wars before the 20th century (non-religious related): 189,400,000.
    Years: ~1,900.
    Deaths/year: 99,680.

    Deaths from wars since the 20th century (non-religious related): 96,600,000.
    Years: 113 (at time of compilation).
    Deaths/year: 854,870.

    Deaths due to Communism: 94,360,000.
    Years: 96 (2013).
    Deaths/year: 982,920.

    Religion-related war deaths: 14,808,500.
    Years (CE/since Christ, 2013): 2013.
    Deaths/year: 7,360.

    2 votes
    1. [2]
      Fal
      Link Parent
      Probably has to do with wars rarely being motivated solely by one cause, such as religion. Off the top of my head, the Taiping Rebellion has a significant religious element, and killed ~25...

      Probably has to do with wars rarely being motivated solely by one cause, such as religion. Off the top of my head, the Taiping Rebellion has a significant religious element, and killed ~25 million, so it clearly isn’t included in the list above as ‘religious’.

      16 votes
      1. Asinine
        Link Parent
        Honestly, it's been so long since I looked it up, I can't tell you how everything sorted out. Even if you added the 25M to the body count, that would raise the "deaths per year" for religions to...

        Honestly, it's been so long since I looked it up, I can't tell you how everything sorted out. Even if you added the 25M to the body count, that would raise the "deaths per year" for religions to around 21k+. Still nothing compared to humans just being dicks and creating war. (Granted, humans being dicks and using religion is definitely more horrible, with the assumption that religion should be solving issues instead of creating more...)

        1 vote
    2. [4]
      skybrian
      Link Parent
      Counting is difficult, especially for historians. Behind every number like that, there is a whole lot of research, lots of uncertainty, and some fairly arbitrary judgement calls about how to count...

      Counting is difficult, especially for historians. Behind every number like that, there is a whole lot of research, lots of uncertainty, and some fairly arbitrary judgement calls about how to count things.

      It might be better to give a range? But that would be arbitrary too.

      And then divide by year? I don't see that making sense. Charting by decade might be more enlightening.

      And then, after all that, there's a question of relevance to whatever question you're trying to answer.

      6 votes
      1. Asinine
        Link Parent
        The whole original question that caused me to look into it back in the day was similar to the OP's post - where we have a lot of data but we also have our own easy biases we've picked up. Trust...

        The whole original question that caused me to look into it back in the day was similar to the OP's post - where we have a lot of data but we also have our own easy biases we've picked up.
        Trust me, I am not defending religion at all, but this seemed similar to me.

        1 vote
      2. [2]
        Asinine
        Link Parent
        I just realized you're the OP. (And I realize I've replied beforehand. But just wanted to re-chime in a bit.) Thing is, yes the count is always difficult. We want video-cam footage of everything,...

        I just realized you're the OP. (And I realize I've replied beforehand. But just wanted to re-chime in a bit.)
        Thing is, yes the count is always difficult. We want video-cam footage of everything, and the stats to rival Google/Amazon or whatever.
        I compiled this 10+ years ago, but I think it was still in that infancy stage where we had the numbers and data coming up but we weren't at where we are now.
        The original reason I posted the deaths per year was because I've never been a huge fan of religion (as the masses are easily plied - my great-uncle whom I idolized for getting a great degree bought thousands of dollars of indulgences to "get out of purgatory free"... but he had pictures of shaking hands with JC Penny, Ronald Regan, and Jimmy Carter...) so, religion was always the "poo poo" and they do crusades and witch hunts and kill all the good people. And it should have been going on since CE/Jesus, which is why I tallied up the years. Then I minored in an option that heavily studied WWI and WWII and exemplified how things had already changed and continued to change, and maybe I dunno, not for the better?

        So, I guess long story short (and if you'd like short story long, please see anything by Dickens :), counting is difficult. That's probably a "I am tired, and I should probably not be posting" response, but maybe I touched better on things. Or not... haha, at least on Reddit, I could post my tired answers and just never bother again. But I'm curious if I at least held up a bit.
        Cheers, it's bedtime.

        1 vote
        1. skybrian
          Link Parent
          I guess the question is, what would you change your mind about based on the body count? What do you hope to learn by researching this stuff? We can use approximate numbers to make rough...

          I guess the question is, what would you change your mind about based on the body count? What do you hope to learn by researching this stuff? We can use approximate numbers to make rough comparisons, but our judgements tend to be based on vibes.

          Historians usually aren't counting at all, they're making estimates, or deciding whether the numbers other people wrote down are plausible. You can't count things that aren't there anymore.

          1 vote
    3. [4]
      NoblePath
      Link Parent
      Does “deaths due to communism” include non-war deaths?

      Does “deaths due to communism” include non-war deaths?

      2 votes
      1. Sodliddesu
        Link Parent
        See, I would prefer a more intense breakdown - I don't want to know "Deaths from Communism" which is basically impossible to really know, I want to know starvation deaths in a certain period. Then...

        See, I would prefer a more intense breakdown - I don't want to know "Deaths from Communism" which is basically impossible to really know, I want to know starvation deaths in a certain period. Then cross reference that with local famines or poor growing sessions.

        Deaths due to religion, another rough metric, is that counting the unmarked graves in Canada and the crusades in one fell swoop? What about Spanish resettlement of southwestern American tribes and Afghan honor killings? It needs to be broken down more, and accurately reported in the first place, before we can even know if they're accurate in the first place.

        Otherwise none of those numbers matter, war time or not.

        5 votes
      2. [2]
        Asinine
        Link Parent
        Actually, it was all non-war deaths. I believe I based it off this link https://en.wikipedia.org/wiki/The_Black_Book_of_Communism#Estimated_number_of_victims. That currently tallies 94 million not...

        Actually, it was all non-war deaths. I believe I based it off this link https://en.wikipedia.org/wiki/The_Black_Book_of_Communism#Estimated_number_of_victims. That currently tallies 94 million not related directly to war.

        1. itdepends
          Link Parent
          You should really read the "Reception" section in Wikipedia, the Black Book of Communism is... not a great source. It does however tie in nicely with the subject at hand in the sense that it...

          You should really read the "Reception" section in Wikipedia, the Black Book of Communism is... not a great source. It does however tie in nicely with the subject at hand in the sense that it appears to also have uncited and unsupported claims.

          Moreover, three of the book's main contributors (Karel Bartosek, Jean-Louis Margolin, and Nicolas Werth)[6] publicly disassociated themselves from Stéphane Courtois' statements in the introduction and criticized his editorial conduct.[35] Margolin and Werth felt that Courtois was "obsessed" with arriving at a total of 100 million killed which resulted in "sloppy and biased scholarship",[43] faulted him for exaggerating death tolls in specific countries,[6][44]: 194 [45]: 123  and rejected the comparison between Communism and Nazism.[3][note 3] Based on the results of their studies, Courtois estimated the total number of the victims at between 65 and 93 million, an unjustified and unclear sum according to Margolin and Werth.[17] In particular, Margolin, who authored the book's chapter on Vietnam, stated that "he has never mentioned a million deaths in Vietnam";[6] Margolin likened Courtois's effort to "militant political activity, indeed, that of a prosecutor amassing charges in the service of a cause, that of a global condemnation of the Communist phenomenon as an essentially criminal phenomenon."[3]

          4 votes