If you read the article (doesn't seem like you did), you'll note that they did isolate the people messing up the data set with statistical analysis. It's in the news because the conclusion that...
If you read the article (doesn't seem like you did), you'll note that they did isolate the people messing up the data set with statistical analysis. It's in the news because the conclusion that they came to was that people were not answering the surveys honestly (not a surprise given results about the validity of surveys in general - not useless, but flawed)
You're assuming that it's a few outliers rather than malice, which may be far as a starting point, but when the researchers doing the study have made a conclusion based on their access to the data & analysis, I'm going to trust them over you. They even give past examples of mischievous responders in the article
The fact that the "outliers" (read: people who lied) were predominantly present among boys aligns with what we'd expect from social norms & socialization.
They are also not necessarily liars, but a deviation from the expected norm whose input can be safely dismissed as untrustworthy
Sure, but these people are liars. The referenced literature in the paper establishes respondents as such & the literature has targeted 'mischievous responders' as a factor. That's the entire point of the study we're talking about: to show that people are lying in order to influence the results.
If there are many – like in the case above – i.e. they exceed a certain percentage (there are standardised ways to detect different types of outliers and compare it to the whole of the sample), the research is flawed in design (and should not be considered a victim of misuse)
They aren't traditional outliers, again. They are people who are purposely answering questions incorrectly in order to influence the results. Even more, the research is most definitely not 'flawed in design' if it gets people who purposefully answer incorrectly, that's just a function of how kids work. These studies show how it can be corrected for.
but I find it discouragingly ridiculous that there was a report on something that shouldn't even have happened
How should it 'not have happened'? Are they supposed to somehow control the people answering the survey so that they don't make up answers?
but the people behind this study who obvously didn't learn enough statistics (which I did).
And yet you're calling me the passive-aggressive one? You're the person talking to actual researchers who had first-hand experience with the data. These people have PhDs. I'm sorry but you're the one being extremely dismissive.
“mischievous responders”—respondents who mislead researchers by providing extreme and untruthful responses to multiple items, perhaps because they find it “funny” to do so
The section on the technique they used to identify the mischievous responders (who are again distinct from traditional outliers - outliers are valid data points but are misrepresentative of the whole population and thus significantly bias results) is long, so I won't quote it here, but it's helpful in understanding why the technique is considered novel. The pop science article, as always, oversimplified the technique.
If you want to read the literature on mischievous responders, which is developing, here are some papers:
It's clear this phenomenon can't simply be reduced to traditional "outliers"
I honestly find it really condescending that you assume you know more than people who went to school for these topics, develop techniques to correct these issues & perform research on correcting these biases in these surveys. These issues have been known in the literature (studies have identified the issue for years - especially in the limitations section of research), exactly why the study was published.
If you read the article (doesn't seem like you did), you'll note that they did isolate the people messing up the data set with statistical analysis. It's in the news because the conclusion that they came to was that people were not answering the surveys honestly (not a surprise given results about the validity of surveys in general - not useless, but flawed)
You're assuming that it's a few outliers rather than malice, which may be far as a starting point, but when the researchers doing the study have made a conclusion based on their access to the data & analysis, I'm going to trust them over you. They even give past examples of mischievous responders in the article
The fact that the "outliers" (read: people who lied) were predominantly present among boys aligns with what we'd expect from social norms & socialization.
Uh, the methods for detecting outliers vary drastically, which is why there are numerous studies that compare methods of detecting them (https://appam.confex.com/appam/2014/webprogram/Paper11278.html & https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4893901/ to start).
Sure, but these people are liars. The referenced literature in the paper establishes respondents as such & the literature has targeted 'mischievous responders' as a factor. That's the entire point of the study we're talking about: to show that people are lying in order to influence the results.
They aren't traditional outliers, again. They are people who are purposely answering questions incorrectly in order to influence the results. Even more, the research is most definitely not 'flawed in design' if it gets people who purposefully answer incorrectly, that's just a function of how kids work. These studies show how it can be corrected for.
How should it 'not have happened'? Are they supposed to somehow control the people answering the survey so that they don't make up answers?
And yet you're calling me the passive-aggressive one? You're the person talking to actual researchers who had first-hand experience with the data. These people have PhDs. I'm sorry but you're the one being extremely dismissive.
Let me quote from the research so you understand the purpose of the study (which you can read here: https://ajph.aphapublications.org/doi/full/10.2105/AJPH.2018.304407):
The section on the technique they used to identify the mischievous responders (who are again distinct from traditional outliers - outliers are valid data points but are misrepresentative of the whole population and thus significantly bias results) is long, so I won't quote it here, but it's helpful in understanding why the technique is considered novel. The pop science article, as always, oversimplified the technique.
If you want to read the literature on mischievous responders, which is developing, here are some papers:
https://journals.sagepub.com/doi/abs/10.3102/0013189X14534297
https://journals.sagepub.com/doi/10.2501/IJMR-54-1-129-145
https://peerj.com/articles/2401/
http://psycnet.apa.org/fulltext/2016-35905-001.html
It's clear this phenomenon can't simply be reduced to traditional "outliers"
I honestly find it really condescending that you assume you know more than people who went to school for these topics, develop techniques to correct these issues & perform research on correcting these biases in these surveys. These issues have been known in the literature (studies have identified the issue for years - especially in the limitations section of research), exactly why the study was published.