English is not..., sorry for that. There are many studies with important conclusions regarding human's health that are presented daily by scientific and non-scientific media. This is a good way to...
English is not..., sorry for that.
There are many studies with important conclusions regarding human's health that are presented daily by scientific and non-scientific media. This is a good way to keeping us well informed and, here is where the problem starts, making "good choices."
Recently, a meta-analysis was posted and discussed on Tildes.net. The more steps, the lesser the mortality. According to the authors, walking more was associated with a longer life span.
If the study conclusion was right and those who did exercise live longer, that does not translate to "one of us" doing exercise and living longer. Why? Because the study compared groups of people and those groups that have a higher average number of daily steps have a smaller risk than other groups that exercised less. Groups, not every person. The results referred to what happened with groups of people who walked more than other groups. The results did not say that every person lived longer.
Unfortunately, many of us get in the ecological fallacy trap and believe that a person who walks more has a lower risk of dying. That has not been part of the study. It was not included. The results cannot answer the research question: what about a person?
We studied people. We concluded about people and we applied the results to a person. That is ecological fallacy.
Many changes in behavior or exposure will benefit only some members of the population. If only benefited a few, that would change the average life span enough to make a difference with other populations that did not change behavior or exposure. The population got better results in average even if for most members the change was irrelevant.
If exercise benefited health in a population study does not imply or rule out that exercise is going to benefit a specific person or even a subgroup of the population unless it was analyzed in the study.
The results saying "the probability of dying increases by 10%" refer to a community and they should not be applied to every member of such community. For every member the experience is that they have or not died. A person does not die a 10% more. A person is alive or not. Assuming that by modifying my individual behavior and exposure to look more like the population with lower mortality rates will extend my life span is a mistake and it can be a costly mistake because what was good for the population can be bad for me. My own life span is not the average life span of people like me and I do not have respond to everything as my population does. Maybe two liters of water per day is too much for me even is that is the average water consumption of a healthy population.
The way to know your own individual risk, not the risk of people like you, is not with population studies but with clinical knowledge. Knowledge about your own anatomy, physiology, clinical history, character traits, gens, etc. We are misled when told "people with this disease has a lower chance of dying... and so those are YOUR chances of dying." That is in people with the disease. The group of people with this disease. That is not my individual case, not me. Will I be in the lower or high end of the distribution? What would be my contribution to the average?
Without clinical studies on my persona and proper following up, the answer will not be provided with population studies. Even if the population of people is diced more and more finely because it will always be the average of a population.
I think you’re misunderstanding the purpose of these studies. These types of studies are about risk factors. A risk factor is really a sort of guess based on population statistics - people who do...
I think you’re misunderstanding the purpose of these studies.
These types of studies are about risk factors. A risk factor is really a sort of guess based on population statistics - people who do x are more at risk of y, or if you do less of x you are at less risk of y. In this case it’s saying “people who walk more are at less risk of dying younger”. And “people” Is defined in each study too - people with diabetes, or people over age 65, or people with family history of heart disease, or whatever group they define - in this way you can start to extrapolate to your own circumstances.
The way science is reported in media isn’t always nuanced like this because it doesn’t drive engagement. So a headline might read “Walk More to Live Longer” even though this isn’t what the actual study said. This is why it can be a good idea, if you know how to properly read studies (which is a skill that not many people get taught, so no shame for anyone who doesn’t have it), to go to the source and see what’s actually been concluded - even just the abstract usually gives a lot more specific information than the reports on it.
Your comment is the type of comment why I posted about ecological fallacy. Unfortunately, it seems I could not communicate the issue of ecological fallacy. The point is not the risk factor...
Your comment is the type of comment why I posted about ecological fallacy.
Unfortunately, it seems I could not communicate the issue of ecological fallacy. The point is not the risk factor associated with the disease or outcome in a given population. The point is that the risk factor does not affect everybody in such population. Therefore, to modify the behavior of an individual because the risk factor was found in the population is misleading, at the individual level. The post was about that fallacy.
There is too much ecological fallacy everywhere and when a study is communicated, even in scientific media it appears again and again.
For example, you say "in this way you can start to extrapolate to your own circumstances." That is, exactly, ecological fallacy. You are wrong extrapolating from the population study to your individual persona, even if you were like the population in which the study was performed. Even if you were part of the population in which the study was performed.
The results are the average of the population. You are not 40 year-old because that is the average age of Tildes' users. You have still your own age. A risk factor for a population is not necessarily a risk factor for you.
Saying that something can lower your risk factor is not the same as saying it will definitely work for each person. What is the alternative? You can’t predict the future. You can’t experiment on...
Saying that something can lower your risk factor is not the same as saying it will definitely work for each person.
What is the alternative? You can’t predict the future. You can’t experiment on yourself, realise something hasn’t helped, then go back in time to try another way. We hypothesise the best ways to lower our risk based on what has worked for population groups.
These studies are also used to inform public health decisions. What has worked in one population group can be assessed to be likely to work in another population group.
I don’t think ecological fallacy fits here. The authors of the study that triggered this post didn’t make causal claims instead of correlative, they didn’t extrapolate, they just stated that within their study more walking was associated with lower mortality. What is fallacious there?
The alternative is clinical work. A clinician treat individuals, not populations. We can predict the future in populations, Very accurately, in fact. Regarding the study that triggered my post, I...
The alternative is clinical work. A clinician treat individuals, not populations. We can predict the future in populations, Very accurately, in fact.
Regarding the study that triggered my post, I saw many comments about commenters aiming to change their own life style. That was the reason.
You mentioned causality. That is not necessarily related to ecological fallacy, but given you mentioned it I want to tell you that I just commented in that post the following regarding the attribution of causality"
In the Discussion section the authors say "Therefore, our analysis demonstrates that ‘more is better’ with respect to step counts in both sexes— irrespective of age and the location where walking takes place. In addition, the results indicate that as little as 4000 steps/day are needed to significantly reduce all-cause mortality, and even fewer steps are required for a significant reduction in CV death. "
That is a clear statement about the authors believing in a causal relationship. They say "4000 steps/day are needed to significantly reduce all-cause mortality." I am sure it skipped the reviewers' attention.
And, about this post, yes, ecological fallacy is present every time we tried to attribute group characteristics to individuals. That is the name of that fallacy and we do that too much with health related information.
You're right the reading studies helps to understand what they claimed to do, but they can still be bad studies and without good statistical training, we aren't likely to spot the flaws. A good...
You're right the reading studies helps to understand what they claimed to do, but they can still be bad studies and without good statistical training, we aren't likely to spot the flaws. A good science reporter will ask other people in the field what they think.
There are some basic problems you can spot yourself, though. Studies often do admit that they only found a correlation if you read them carefully, and the press and general public will often ignore that and assume causation.
Figuring out how to deduce a cause, rather than a correlation, can be difficult. Good things in life are often correlated. So are bad things. Problems sometimes cascade. This makes it easy to find correlations. But for any correlation, the cause could be reversed, or a different cause altogether.
For example, healthy people exercise more. But this could be because exercise improves health, or because sick people find it hard to exercise. The underlying cause of not exercising may be some other health problem. If someone who normally exercises stops, it may be a bad sign.
So sure, people who walk more may live longer, but maybe that's because people who get a disease become weak and walk less?
A surprising correlation is that losing weight is often associated with poor health, because unexplained weight loss is more often caused by some health condition than by someone improving their habits.
(None of this is about ecological fallacies though. Those are different statistical problems.)
Good points. Sometimes it is difficult or impossible to perform high quality clinical trials and the only thing left is observational studies. Policy decisions (for example, authorization to...
Good points. Sometimes it is difficult or impossible to perform high quality clinical trials and the only thing left is observational studies.
Policy decisions (for example, authorization to market a drug or building exercise-friendly communities) based on observational studies needs to use the best available evidence without jumping to "we know the cause and so... and then this is the intervention based on causal evidence." Researchers need to be humbler than that.
It is not about ecological fallacies. Well taken.
What you say is what it is usually said, that if you practice healthy habits you will live longer/better/etc. What is the evidence for that? A population study. It works for groups, not for an...
What you say is what it is usually said, that if you practice healthy habits you will live longer/better/etc. What is the evidence for that? A population study. It works for groups, not for an individual.
There are simple computations that show this fallacy in action: Population Attributable Risk or Number Needed to Treat are the most used.
Easier to understand, Number Needed to Treat refers to how many people needs to be treated with a given therapy to get the benefit of the therapy. For example, in https://www.jwatch.org/na48372/2019/02/06/aspirin-primary-prevention-new-meta-analysis it was found that 265 people had to be treated with aspirin to prevent one cardiovascular outcome. That is a group effect. 264 people took aspirin and there was no change in their outcome.
Who was that person? No way to know. It did work? Yes. At the group level.
Then, an individual cannot know whether it will live longer than him/herself by him/her improving lifestyle. The group outcome will improve because the benefit will occur in some members of the group. No way to know, so far, in whom.
*I did not get the "pedantic hypotheticals" thing.
What op and linked article are saying is that you cannot draw conclusions about an individual person, and their likelihood of benefit (or detriment), based on a population level study. And that’s...
What op and linked article are saying is that you cannot draw conclusions about an individual person, and their likelihood of benefit (or detriment), based on a population level study. And that’s mathematically true according to Wikipedia.
The upshot is that it’s not rational to recommend or make changes to individual (or subgroups) based on the population study results.
Speculating here, but for example, some in the population may have a myocardial defect such that their lifespan is shotened by exercise. But that’s almost irrelevant. The point is it is mathematically false to draw any conclusion about individuals from a group study data.
I would think group study data could set the stage for further individual study, though.
Edit: also this is talking about studies of populations, where x% of some individuals in a population show some characteristic. As opposed to say a randomized controlled study, which is studying a group of individuals, individually.
Very well said. However, the ecological fallacy also applies to randomized controlled studies (RCT). In fact, the number needed to treat is originated on RCT as an acknowledgment that what worked...
Very well said.
However, the ecological fallacy also applies to randomized controlled studies (RCT). In fact, the number needed to treat is originated on RCT as an acknowledgment that what worked in the intervertion group did not work in everybody part of the intervention group.
Can you link back to the original thread? Because I think what you're claiming here hinges quite a bit on the details of the study, and how it was presented. For example, if they studied 3 ethnic...
Can you link back to the original thread? Because I think what you're claiming here hinges quite a bit on the details of the study, and how it was presented. For example, if they studied 3 ethnic groups, and in each group they observed a benefit, then there's no reason to assume that any one person won't benefit, unless we've also done studies that further separate the population into different groups that does show that. Absent evidence that goes into those groups, where there is no benefit for at least some groups, or where group assignment can be assumed to change based on steps walked, I don't think you can reasonably reject the conclusion that at least statistically, that benefit applies to the individual. But again, hinges on the details of the study.
I do not know how to properly link. https://tildes.net/~health/197f/worlds_largest_study_shows_more_you_walk_lower_your_risk_of_death Anyway, that study is not relevant for this post. Their...
I do not know how to properly link. https://tildes.net/~health/197f/worlds_largest_study_shows_more_you_walk_lower_your_risk_of_death
Anyway, that study is not relevant for this post. Their comments are relevant. That study has problems with causality assumptions. The commenters has the problem with ecological fallacy.
Regarding ecological fallacy, let's imagine two identical populations of 1,000 people each.
In each population, 20 people die every year due to cardiovascular diseases. Their rate of death due to cardiovascular diseases is 2%. This has happened for many many years.
A new medicine or procedure or life style change is tried in one population, the other is used as reference or control. Everybody in the intervention population, one thousand, received the treatment.
After the intervention, in the reference population 20 people die as usual. In the treated population only 15 people die.
The intervention had an effect in five individuals. 995 were treated and there was not visible results in them. Only five had their life changed for the better.
The scientific article communicating about this study is going to say that there is an effect of the intervention, simplifying: 15/20 = .75 or 25% reduction in the rate of death.
That does not mean that everyone in the treated population saw a reduction in their own individual mortality. In fact, 995 had no change at all. 15 of them died, as they were going to anyway, and 980 continued living, as they were going to anyway. The effect was in 5 individuals. And the effect on those five individual was enough to make a difference in the rate of the population.
The ecological fallacy is to believe that because there was a reduction of 25% in the mortality rate, my own mortality rate will be reduced in 25% or that the intervention will benefit me.
I hope this helps. I think I should have posted a numerical example earlier.
What you're describing there does not seem like a ecological fallacy there. That's just miscommunication about the size of the effect. In your example, the reduction in rate of death (or at least...
Exemplary
What you're describing there does not seem like a ecological fallacy there. That's just miscommunication about the size of the effect. In your example, the reduction in rate of death (or at least rate of death due to cardiovascular disease) was indeed 25%. Not 25 percentage points, but 25%. From 2% to 1.5% is a 25% reduction, or a 0.5 percentage point reduction. That is a very common stumbling stone, but not nearly a fallacy.
And it is absolutely valid to say that everyone in the treated group experienced a 25% reduction in rate of death. That only 20 of them would've died means that the 25% decrease only had an actual effect (i.e. not dying) in 5 people, or 0.5%. That's how that works.
The fact that other factors might lead to people walking less (well, in the original study, and not in our hypothetical RCT) and also lead to those people dying more is an issue with causality; I don't see an ecological fallacy there. Now, of course it's important to be mindful of possible confounders of causality like the above, and from an epistemological perspective this should spur on RCTs to study the effect of the intervention of "telling people to walk more" to nail down how much of the effect is causal.
Let me add a modification to your example to construct what I'd accept to be an ecological fallacy. Let's say of those 1000 people in each cohort, 500 are desk workers and 500 are marathon runners, who will experience cardiovascular death by overexertion if they walk any more than they already do. Let's say they account for 5 deaths of the 20 (because they're in good health) and that they account for 10 deaths in the intervention group. So, control group has 15 dead desk workers and 5 dead marathon runners, while the intervention has, let's fudge the numbers to stay consistent, 10 dead marathon runners and 5 dead desk workers. If we have all of that information, it would be an ecological fallacy to say that according to the first RCT (yours) which did not differentiate between marathon runners and desk workers, everyone gets a 25% risk reduction if they just walk more; because we know that that group splits into two distinct subgroups which react differently. If you're a marathon runner, you'll get a 100% increase in risk, and as a desk worker you'll get 66% reduction in risk. I can see calling that a ecological fallacy.
(Interestingly, doing math with those changes to risk is another good example of an ecological fallacy. One would assume that since the groups are equally large, I can just add the +100% and the -66% and get +33%, thus an increase in risk across the population. But deaths went down from 20 to 15. What gives? Well, the math's wrong because those relative risks we added actually refer to percentages of different things: Of the 15 and 5 dead in the control group.)
What I can't see is hypothesizing that differences within the studied group must exist, therefore we don't know anything. That seems to be awfully close to calling the entire field of statistics fallacious. Because I can always find another axis along which to separate the distribution. What about female vs male marathon runners? What about male marathon runners that prefer clockwise tracks vs male marathon runners that prefer counter-clockwise tracks? What about male left-handed marathon runners that prefer clockwise tracks vs male right-handed marathon runners that prefer clockwise tracks? I can always conjecture that the trend will invert in such a subgroup, and therefore the 100% increase in harm does not apply to me, because I'm special. But I need some kind of justification for that, don't I? Ideally, I have a high-quality study to prove it, in which case I would agree that sticking to the more general prediction is an ecological fallacy. Maybe I have a mechanistic explanation or anecdotal evidence to indicate that maybe the results don't generalize to a certain subgroup, and therefore I should maybe work towards acquiring that study. But if I have nothing, I have to assume that the results do generalize, because not accepting that then means not accepting that ever, because the effect might just be the other way around in the case of male right-handed marathon runners that prefer clockwise tracks.
In that sense maybe I'm a bit at odds with Wikipedia, because in the absence of good evidence I have to assume that the effects we observed on the larger group apply equally across all subgroups. It's an assumption alright, and probably one of the first sources of error we should check if things don't work out the way we want them to; and maybe we shouldn't rely on it to begin with if the stakes are too high. But in the absence of further evidence, it is the most likely explanation.
That said, some amount of doubt is in order, particularly if we're talking about simple studies and complex systems. In that situation, we're bound to miss subtleties. And I guess I'm unclear what your overall thesis here is. Is it "we don't know anything until we've studied male left-handed marathon runners that prefer clockwise tracks vs male right-handed marathon runners that prefer clockwise tracks" (really needs an acronym) or is it "if you know you're a marathon runner, you might want to use that bit of information when estimating the health benefits of walking more"?
Part by part. There is a 25% reduction in risk (a ratio) and .5 reduction in risk (a difference). One is in the multiplicative scale and the other is in the linear (additive) scale. Both are...
Part by part.
There is a 25% reduction in risk (a ratio) and .5 reduction in risk (a difference). One is in the multiplicative scale and the other is in the linear (additive) scale. Both are right.
I do not understand the logic of your statement "And it is absolutely valid to say that everyone in the treated group experienced a 25% reduction in rate of death." I mean, how can I understand that my own probability of dying decreased because others did not die? There is no evidence for that. That is ecological fallacy at its best.
What you are involving by making the distinction between marathoners and desk workers is what we know as subgroups (a third variable) and they can act as effect modifiers or as confounders. In your example, you used that variable as an effect modifier: the effect of walking on mortality will be different across subgroups.
3.5) Another thing you did with your example is that you moved from the probabilistic approach when studying populations to a deterministic approach (100% of marathoners). Empirically, if something happens 100% of the time is not part of a study because it is already common knowledge. It follows the laws of nature, like the effect of gravity. When you switch to a deterministic approach two things are evident a)The deterministic trick is a straw man fallacy by itself b) The deterministic trick open the door to interpretations like "I did smoke all my life and I am in my 90's, therefore smoking does not kill...," which are correct at the individual experience level ("does not kill ME") and wrong at the population level (smoking increases the rate of mortality).
Effect modification is the name for the effect in the larger group varying by subgroups. To deny that possibility is falling again into ecological fallacy and it is, paradoxically, frequent in ecological studies!!
The overall point is that the effects vary within groups and the variability is such that it is wrong, very wrong, to assume that the effects on a population represent the effects on each member of such population.
I'm not sure I have either the patience or the words to communicate about this entire matter effectively. A bunch of what you said strikes me as a very different (from mine) conceptualizing of...
I'm not sure I have either the patience or the words to communicate about this entire matter effectively. A bunch of what you said strikes me as a very different (from mine) conceptualizing of chance and probability, or drastically different words to express that conceptualization, also considering the "english is not my first language" part. It could also be an insufficient understanding of statistics but I'm going to assume it's not, and try and clean up as many possible misunderstandings as I reasonably can. If an innocent bystander (you, dear reader) happens to understand both of us well, and help in untangling wires is appreciated.
No contest. That was exactly my point.
Let me put it like this: You roll a die. It lands on 6. What was the chance of you rolling a 1, assuming a fair die? I hope you agree it's 1/6, and not 0. If you say the chance was 0, because ultimately it ended up on a different number, then I understand why you'd say that the 25% reduction makes no sense on an individual level. But same as a dice roll, whether you die is in the above scenario ultimately random. And by walking more, you can affect the probability of that random event. Therefore, your probability of dying decreased. It is possible that there is no causal link in your case between walking more and dying less, because you're a left handed marathon man. But since we're not observing at that level of detail, and hence have no information how your left handedness interacts with your walking and health, we subsume it as randomness. Consider also that the physics of how the dice rolls is theoretically predictable, so it ultimately is a deterministic process (save for some potential marginal truly random quantum effects), but we don't conceptualize it as such because no one has time to model a dice roll to such a level of detail as to make it deterministic. From that perspective, while there might be a theoretical deterministic chain of events from initial conditions to you dying or not, we talk about risks, because no one resolves or understands reality to such a degree of detail as to make that determinism useful. Hence: We speak in probabilities, and we resolve those probabilities as fine-grainedly as the available information supports. Put very very simply: How do you know the intervention did not save you? If you don't know, you must admit that probability is the right terminology to talk about the effect.
the effect of walking on mortality will be different across subgroups.
No. It can be different. But for example, it's extremely unlikely that left-or right-handed marathon runners' cardiovascular systems react differently to walking a bit more. I could come up with any number of subgroups, most of which completely irrelevant (i.e. statistically independent from what we're interested in), until most of my subgroups have n=0. Then what? Do we assume that their effects are all different, and therefore we learned exactly nothing from the original trial? Alright, if you insist, but then we have to study each of these 8 billion subgroups, but we're only going to get shitty group sizes, so we won't learn anything either. Therefore, statistics is thoroughly impossible. Or we put a prior on it (i.e. make the assumption) that any subgroup is independent of the effect we're looking for. We're basically saying "unless evidence to the contrary exists, we assume that left-handed and right-handed marathon runners react the same to our intervention." Repeating my last, we can inject here some mechanistic or other evidence that need not be statistical to cast doubt on that assumption. For example, even from the original RCT, it seems plausible that marathon runners might react differently. I'd rate that kind of evidence as at least somewhat valid. The reason I'm beating this horse again is that if you call "ecological fallacy" every chance you get, statistical inference becomes impossible, which is a notion we should speedily reject. We should hold on to that independence assumption until compelled to drop it, because that's all that stands between our valued inference and the chaos of the world. And because there's always going to be infinitely more irrelevant group splits than relevant ones, I'd say it's not an unreasonable assumption. Most subgroups will not make a difference, therefore it is madness to assume such a difference without further evidence beyond the existence of subgroups.
3.5: There is no determinism I invoked. The 100% is a 100% in relative terms, i.e. when 5 marathon runners died in the control, 10 did in the intervention, therefore a 100% relative increase. The absolute increase is 1PP.
To clarify: I'm not denying that effects can vary by subgroups, quite the contrary. It can and will happen. But absent any evidence for which subgroups to look at, amongst the infinitely many different axes you could use to separate, you should assume that any individual subgroup split has no effect, while being mindful that there are some for which that assumption is wrong, but you don't know which ones that are.
I understand. Depending on how exactly you mean that, it's either in conflict with my assertion that you should assume that each split is irrelevant, or it isn't. I guess the essential question then is whether you apply any kind of "presumption of irrelevance", or do some other kind of filtering, and if so, how. I.e., when you read the study in the previous-linked article, do you simply say "I'm different from the aggregate population they studied", or do you say "but I live at 4000m above sea level, therefore I have reason to believe my cardiovascular system reacts differently to exercise than that of the aggregate population they studied"? And how do you decide whether "4000m above sea level" for example is sufficient difference to discount the results? Or that "but I'm left-handed" is insufficient?
I hope my reply is even half as lucid as I think it is.
I do agree, it is hard to communicate. We do agree in many points and we have disagreements in a few. The rolling a die is a clear example of ecological fallacy. Let's see whether can share some...
I do agree, it is hard to communicate. We do agree in many points and we have disagreements in a few.
The rolling a die is a clear example of ecological fallacy. Let's see whether can share some common understanding.
What is going to be the next roll? It can be any from one to six. Individually, I do care that it will be not 1 (dead), it can be any of the others (alive). What will be the next one? It cannot be answered with 5/6 chances of being not one because that was not the question. The next roll will be one or will be not one and, therefore the answer should be "it will be one" or "it will not be one." The question at the individual level can be answered by, as you say, having (almost) complete knowledge of all factors involved in that roll. In that roll, not in a series of rolls from where the uniform, and the 1/6 and 5/6, probabilities were obtained. The answer providing probabilities of the next event works very well rolling the dice many times. It answers a different question. Not my question about the next roll.
Now, if I walk more, will my health be better? According to a study, those who walk more had higher chances of being healthy. But, I am asking about myself because I want to decide whether to walk more or not. Let's say that I was part of the study and I do not have an answer because there is no control group for me. There is a control group for the group I belong to, not for me. Therefore, answering my question about me with the chances of the group is like predicting the next roll based on probabilities obtained with a series of rolls in which I am not interested because I want to know the answer about me.
The only way to be right predicting individual outcomes with group probabilities is predicting multiple individual outcomes, which is not an individual prediction but a group prediction. Some will be correct prediction and some will be wrong predictions and the average likely will be the probabilities of the initial study, the series of rolls.
Poker players play using statistics and they know that if they played multiple times according to those probabilities at the end they will make some money. But they cannot know whether they will make money in each individual play.
The answer about individual prediction cannot come from epidemiological studies but for clinical work. 80% of long term smokers do not die of lung cancer although their risk is increased 30 times in comparison with non-smokers. How can you explain that except by saying that the increased risk is for the population, not for each smoker. Why? The answer is we do not know yet. We pretend we know to make those who smoke a lot to quit or smoke less, to save the life of at least some of those who would have died due to lung cancer.
Yes, I suspect this might be true of lots of drugs? Strong effects for some but no effect for others. Maybe it would be clearer to explain it as people varying rather than an "ecological fallacy"...
Yes, I suspect this might be true of lots of drugs? Strong effects for some but no effect for others. Maybe it would be clearer to explain it as people varying rather than an "ecological fallacy" which is rather abstract?
An example might be psychiatrists who will try different drugs until they find one that works.
But I'm not sure what that means other than setting expectations about needing to do experiments? Trying a different drug is an n=1 experiment.
It's expensive, though, and requires persistence. Figuring out cheap tests to see if someone will benefit can improve the odds. It may be difficult to find a test that's as good as actually trying it.
More cynically, though, drug companies benefit from people taking drugs because they might help. Having a good, cheap test to tell you that a drug won't work for you would reduce market size, perhaps drastically, turning it into a specialty drug.
I do agree with you that it might be clearer explaining as variation in response and not with ecological fallacy, which is, as you say, exploited by pharmaceutical companies. Clinical work (I go...
I do agree with you that it might be clearer explaining as variation in response and not with ecological fallacy, which is, as you say, exploited by pharmaceutical companies.
Clinical work (I go to my nurse practitioner) is N=n=1 and my nurse study me repeated times and try what works.
Currently, clinical work is a mix of epidemiology and knowledge about the individual patient. What patients demand is what is going on with me (diagnosis) and what will happen to me (prognosis). What public health institutions want is what is going on and what will happen with populations, groups of people.
Clinical work is, as you say, very expensive. We are very far from individualized medicine.
It's just as much of an error to say "this correlative population study means that X will certainly have Y effect for each individual" as it is to say "the fact that this was a correlative...
It's just as much of an error to say "this correlative population study means that X will certainly have Y effect for each individual" as it is to say "the fact that this was a correlative population study means that we can't draw any conclusions at all."
Studies suggest that statins reduce the risk of mortality for those at high risk of heart disease. That doesn't mean that they will work for everyone in that population, but if you're at high risk for heart disease, you should take statins.
This. The ecological fallacy is a thing but OP seems to be over-extending the concept to conclude that it's impossible to ever draw conclusions from studies. You could use their same arguments to...
This. The ecological fallacy is a thing but OP seems to be over-extending the concept to conclude that it's impossible to ever draw conclusions from studies. You could use their same arguments to conclude that smoking doesn't cause lung cancer (and tobacco companies have used that argument in the past to avoid regulation).
It's like, yeah, I can't guarantee something about an individual based on these studies, but that doesn't mean statistics just cease to apply to them. And as for the only way to draw conclusions being working with a clinician... how do you think the clinician learns how certain things manifest and what to look out for? Unless they have precognition, they're taking some level of statistical risk factors into consideration when they assess your condition because those things can affect how they interpret their observations.
That is wrong, the number needed to treat for statins hoover around 200-300, implying that to benefit one patient, the one that will improve his/her health with statins, 200 or 300 have to be...
That is wrong, the number needed to treat for statins hoover around 200-300, implying that to benefit one patient, the one that will improve his/her health with statins, 200 or 300 have to be treated and receive no benefit. Physicians know that and prescribe statins because if they do not that one lucky person will never receive the good effect.
We take statins, and every medicine, with a grain of salt.
That seems inherent in "risk of mortality" though? We are doing probability calculations because we don't know what will happen and outcomes vary. Given that, I don't see anything wrong. It seems...
That seems inherent in "risk of mortality" though? We are doing probability calculations because we don't know what will happen and outcomes vary.
Given that, I don't see anything wrong. It seems good to emphasize our ignorance about what will happen, though.
Yes. With any rate, mortality included. Probability calculations because at the group level its members vary randomly. Nothing wrong with that. The fallacy is to assume that the probability of the...
Yes. With any rate, mortality included.
Probability calculations because at the group level its members vary randomly. Nothing wrong with that.
The fallacy is to assume that the probability of the group is the probability of each individual member of the group. We do not know what will happen at the individual level and we have a good idea of what will happen at the group level.
I do agree. Well said. Association is necessary for causality. Just not enough to assert it. Added: from a public health point of view, it would be cringe to say "stop smoking because is going to...
I do agree. Well said. Association is necessary for causality. Just not enough to assert it.
Added: from a public health point of view, it would be cringe to say "stop smoking because is going to kill some of you, a few... maybe 10-20% of long term smokers" (https://www.sciencedaily.com/releases/2022/04/220411113733.htm). In fact, most smokers do not get lung cancer and, despite that, epidemiologically smoking causes lung cancer.
English is not..., sorry for that.
There are many studies with important conclusions regarding human's health that are presented daily by scientific and non-scientific media. This is a good way to keeping us well informed and, here is where the problem starts, making "good choices."
Recently, a meta-analysis was posted and discussed on Tildes.net. The more steps, the lesser the mortality. According to the authors, walking more was associated with a longer life span.
If the study conclusion was right and those who did exercise live longer, that does not translate to "one of us" doing exercise and living longer. Why? Because the study compared groups of people and those groups that have a higher average number of daily steps have a smaller risk than other groups that exercised less. Groups, not every person. The results referred to what happened with groups of people who walked more than other groups. The results did not say that every person lived longer.
Unfortunately, many of us get in the ecological fallacy trap and believe that a person who walks more has a lower risk of dying. That has not been part of the study. It was not included. The results cannot answer the research question: what about a person?
We studied people. We concluded about people and we applied the results to a person. That is ecological fallacy.
Many changes in behavior or exposure will benefit only some members of the population. If only benefited a few, that would change the average life span enough to make a difference with other populations that did not change behavior or exposure. The population got better results in average even if for most members the change was irrelevant.
If exercise benefited health in a population study does not imply or rule out that exercise is going to benefit a specific person or even a subgroup of the population unless it was analyzed in the study.
The results saying "the probability of dying increases by 10%" refer to a community and they should not be applied to every member of such community. For every member the experience is that they have or not died. A person does not die a 10% more. A person is alive or not. Assuming that by modifying my individual behavior and exposure to look more like the population with lower mortality rates will extend my life span is a mistake and it can be a costly mistake because what was good for the population can be bad for me. My own life span is not the average life span of people like me and I do not have respond to everything as my population does. Maybe two liters of water per day is too much for me even is that is the average water consumption of a healthy population.
The way to know your own individual risk, not the risk of people like you, is not with population studies but with clinical knowledge. Knowledge about your own anatomy, physiology, clinical history, character traits, gens, etc. We are misled when told "people with this disease has a lower chance of dying... and so those are YOUR chances of dying." That is in people with the disease. The group of people with this disease. That is not my individual case, not me. Will I be in the lower or high end of the distribution? What would be my contribution to the average?
Without clinical studies on my persona and proper following up, the answer will not be provided with population studies. Even if the population of people is diced more and more finely because it will always be the average of a population.
I think you’re misunderstanding the purpose of these studies.
These types of studies are about risk factors. A risk factor is really a sort of guess based on population statistics - people who do x are more at risk of y, or if you do less of x you are at less risk of y. In this case it’s saying “people who walk more are at less risk of dying younger”. And “people” Is defined in each study too - people with diabetes, or people over age 65, or people with family history of heart disease, or whatever group they define - in this way you can start to extrapolate to your own circumstances.
The way science is reported in media isn’t always nuanced like this because it doesn’t drive engagement. So a headline might read “Walk More to Live Longer” even though this isn’t what the actual study said. This is why it can be a good idea, if you know how to properly read studies (which is a skill that not many people get taught, so no shame for anyone who doesn’t have it), to go to the source and see what’s actually been concluded - even just the abstract usually gives a lot more specific information than the reports on it.
Your comment is the type of comment why I posted about ecological fallacy.
Unfortunately, it seems I could not communicate the issue of ecological fallacy. The point is not the risk factor associated with the disease or outcome in a given population. The point is that the risk factor does not affect everybody in such population. Therefore, to modify the behavior of an individual because the risk factor was found in the population is misleading, at the individual level. The post was about that fallacy.
There is too much ecological fallacy everywhere and when a study is communicated, even in scientific media it appears again and again.
For example, you say "in this way you can start to extrapolate to your own circumstances." That is, exactly, ecological fallacy. You are wrong extrapolating from the population study to your individual persona, even if you were like the population in which the study was performed. Even if you were part of the population in which the study was performed.
The results are the average of the population. You are not 40 year-old because that is the average age of Tildes' users. You have still your own age. A risk factor for a population is not necessarily a risk factor for you.
Saying that something can lower your risk factor is not the same as saying it will definitely work for each person.
What is the alternative? You can’t predict the future. You can’t experiment on yourself, realise something hasn’t helped, then go back in time to try another way. We hypothesise the best ways to lower our risk based on what has worked for population groups.
These studies are also used to inform public health decisions. What has worked in one population group can be assessed to be likely to work in another population group.
I don’t think ecological fallacy fits here. The authors of the study that triggered this post didn’t make causal claims instead of correlative, they didn’t extrapolate, they just stated that within their study more walking was associated with lower mortality. What is fallacious there?
The alternative is clinical work. A clinician treat individuals, not populations. We can predict the future in populations, Very accurately, in fact.
Regarding the study that triggered my post, I saw many comments about commenters aiming to change their own life style. That was the reason.
You mentioned causality. That is not necessarily related to ecological fallacy, but given you mentioned it I want to tell you that I just commented in that post the following regarding the attribution of causality"
In the Discussion section the authors say "Therefore, our analysis demonstrates that ‘more is better’ with respect to step counts in both sexes— irrespective of age and the location where walking takes place. In addition, the results indicate that as little as 4000 steps/day are needed to significantly reduce all-cause mortality, and even fewer steps are required for a significant reduction in CV death. "
That is a clear statement about the authors believing in a causal relationship. They say "4000 steps/day are needed to significantly reduce all-cause mortality." I am sure it skipped the reviewers' attention.
And, about this post, yes, ecological fallacy is present every time we tried to attribute group characteristics to individuals. That is the name of that fallacy and we do that too much with health related information.
Here's an interesting series of blog posts about how sufficiently motivated people can do their own experiments:
Slime Mold Time Mold: N=1
That is so great. Case studies are the epitome of clinical work.
You're right the reading studies helps to understand what they claimed to do, but they can still be bad studies and without good statistical training, we aren't likely to spot the flaws. A good science reporter will ask other people in the field what they think.
There are some basic problems you can spot yourself, though. Studies often do admit that they only found a correlation if you read them carefully, and the press and general public will often ignore that and assume causation.
Figuring out how to deduce a cause, rather than a correlation, can be difficult. Good things in life are often correlated. So are bad things. Problems sometimes cascade. This makes it easy to find correlations. But for any correlation, the cause could be reversed, or a different cause altogether.
For example, healthy people exercise more. But this could be because exercise improves health, or because sick people find it hard to exercise. The underlying cause of not exercising may be some other health problem. If someone who normally exercises stops, it may be a bad sign.
So sure, people who walk more may live longer, but maybe that's because people who get a disease become weak and walk less?
A surprising correlation is that losing weight is often associated with poor health, because unexplained weight loss is more often caused by some health condition than by someone improving their habits.
(None of this is about ecological fallacies though. Those are different statistical problems.)
Good points. Sometimes it is difficult or impossible to perform high quality clinical trials and the only thing left is observational studies.
Policy decisions (for example, authorization to market a drug or building exercise-friendly communities) based on observational studies needs to use the best available evidence without jumping to "we know the cause and so... and then this is the intervention based on causal evidence." Researchers need to be humbler than that.
It is not about ecological fallacies. Well taken.
What you say is what it is usually said, that if you practice healthy habits you will live longer/better/etc. What is the evidence for that? A population study. It works for groups, not for an individual.
There are simple computations that show this fallacy in action: Population Attributable Risk or Number Needed to Treat are the most used.
Easier to understand, Number Needed to Treat refers to how many people needs to be treated with a given therapy to get the benefit of the therapy. For example, in https://www.jwatch.org/na48372/2019/02/06/aspirin-primary-prevention-new-meta-analysis it was found that 265 people had to be treated with aspirin to prevent one cardiovascular outcome. That is a group effect. 264 people took aspirin and there was no change in their outcome.
Who was that person? No way to know. It did work? Yes. At the group level.
Then, an individual cannot know whether it will live longer than him/herself by him/her improving lifestyle. The group outcome will improve because the benefit will occur in some members of the group. No way to know, so far, in whom.
*I did not get the "pedantic hypotheticals" thing.
*typo
What op and linked article are saying is that you cannot draw conclusions about an individual person, and their likelihood of benefit (or detriment), based on a population level study. And that’s mathematically true according to Wikipedia.
The upshot is that it’s not rational to recommend or make changes to individual (or subgroups) based on the population study results.
Speculating here, but for example, some in the population may have a myocardial defect such that their lifespan is shotened by exercise. But that’s almost irrelevant. The point is it is mathematically false to draw any conclusion about individuals from a group study data.
I would think group study data could set the stage for further individual study, though.
Edit: also this is talking about studies of populations, where x% of some individuals in a population show some characteristic. As opposed to say a randomized controlled study, which is studying a group of individuals, individually.
Very well said.
However, the ecological fallacy also applies to randomized controlled studies (RCT). In fact, the number needed to treat is originated on RCT as an acknowledgment that what worked in the intervertion group did not work in everybody part of the intervention group.
It works for groups and people. Just not necessarily for an individual person. Just not necessarily for most people.
Can you link back to the original thread? Because I think what you're claiming here hinges quite a bit on the details of the study, and how it was presented. For example, if they studied 3 ethnic groups, and in each group they observed a benefit, then there's no reason to assume that any one person won't benefit, unless we've also done studies that further separate the population into different groups that does show that. Absent evidence that goes into those groups, where there is no benefit for at least some groups, or where group assignment can be assumed to change based on steps walked, I don't think you can reasonably reject the conclusion that at least statistically, that benefit applies to the individual. But again, hinges on the details of the study.
I do not know how to properly link. https://tildes.net/~health/197f/worlds_largest_study_shows_more_you_walk_lower_your_risk_of_death
Anyway, that study is not relevant for this post. Their comments are relevant. That study has problems with causality assumptions. The commenters has the problem with ecological fallacy.
Regarding ecological fallacy, let's imagine two identical populations of 1,000 people each.
In each population, 20 people die every year due to cardiovascular diseases. Their rate of death due to cardiovascular diseases is 2%. This has happened for many many years.
A new medicine or procedure or life style change is tried in one population, the other is used as reference or control. Everybody in the intervention population, one thousand, received the treatment.
After the intervention, in the reference population 20 people die as usual. In the treated population only 15 people die.
The intervention had an effect in five individuals. 995 were treated and there was not visible results in them. Only five had their life changed for the better.
The scientific article communicating about this study is going to say that there is an effect of the intervention, simplifying: 15/20 = .75 or 25% reduction in the rate of death.
That does not mean that everyone in the treated population saw a reduction in their own individual mortality. In fact, 995 had no change at all. 15 of them died, as they were going to anyway, and 980 continued living, as they were going to anyway. The effect was in 5 individuals. And the effect on those five individual was enough to make a difference in the rate of the population.
The ecological fallacy is to believe that because there was a reduction of 25% in the mortality rate, my own mortality rate will be reduced in 25% or that the intervention will benefit me.
I hope this helps. I think I should have posted a numerical example earlier.
What you're describing there does not seem like a ecological fallacy there. That's just miscommunication about the size of the effect. In your example, the reduction in rate of death (or at least rate of death due to cardiovascular disease) was indeed 25%. Not 25 percentage points, but 25%. From 2% to 1.5% is a 25% reduction, or a 0.5 percentage point reduction. That is a very common stumbling stone, but not nearly a fallacy.
And it is absolutely valid to say that everyone in the treated group experienced a 25% reduction in rate of death. That only 20 of them would've died means that the 25% decrease only had an actual effect (i.e. not dying) in 5 people, or 0.5%. That's how that works.
The fact that other factors might lead to people walking less (well, in the original study, and not in our hypothetical RCT) and also lead to those people dying more is an issue with causality; I don't see an ecological fallacy there. Now, of course it's important to be mindful of possible confounders of causality like the above, and from an epistemological perspective this should spur on RCTs to study the effect of the intervention of "telling people to walk more" to nail down how much of the effect is causal.
Let me add a modification to your example to construct what I'd accept to be an ecological fallacy. Let's say of those 1000 people in each cohort, 500 are desk workers and 500 are marathon runners, who will experience cardiovascular death by overexertion if they walk any more than they already do. Let's say they account for 5 deaths of the 20 (because they're in good health) and that they account for 10 deaths in the intervention group. So, control group has 15 dead desk workers and 5 dead marathon runners, while the intervention has, let's fudge the numbers to stay consistent, 10 dead marathon runners and 5 dead desk workers. If we have all of that information, it would be an ecological fallacy to say that according to the first RCT (yours) which did not differentiate between marathon runners and desk workers, everyone gets a 25% risk reduction if they just walk more; because we know that that group splits into two distinct subgroups which react differently. If you're a marathon runner, you'll get a 100% increase in risk, and as a desk worker you'll get 66% reduction in risk. I can see calling that a ecological fallacy.
(Interestingly, doing math with those changes to risk is another good example of an ecological fallacy. One would assume that since the groups are equally large, I can just add the +100% and the -66% and get +33%, thus an increase in risk across the population. But deaths went down from 20 to 15. What gives? Well, the math's wrong because those relative risks we added actually refer to percentages of different things: Of the 15 and 5 dead in the control group.)
What I can't see is hypothesizing that differences within the studied group must exist, therefore we don't know anything. That seems to be awfully close to calling the entire field of statistics fallacious. Because I can always find another axis along which to separate the distribution. What about female vs male marathon runners? What about male marathon runners that prefer clockwise tracks vs male marathon runners that prefer counter-clockwise tracks? What about male left-handed marathon runners that prefer clockwise tracks vs male right-handed marathon runners that prefer clockwise tracks? I can always conjecture that the trend will invert in such a subgroup, and therefore the 100% increase in harm does not apply to me, because I'm special. But I need some kind of justification for that, don't I? Ideally, I have a high-quality study to prove it, in which case I would agree that sticking to the more general prediction is an ecological fallacy. Maybe I have a mechanistic explanation or anecdotal evidence to indicate that maybe the results don't generalize to a certain subgroup, and therefore I should maybe work towards acquiring that study. But if I have nothing, I have to assume that the results do generalize, because not accepting that then means not accepting that ever, because the effect might just be the other way around in the case of male right-handed marathon runners that prefer clockwise tracks.
In that sense maybe I'm a bit at odds with Wikipedia, because in the absence of good evidence I have to assume that the effects we observed on the larger group apply equally across all subgroups. It's an assumption alright, and probably one of the first sources of error we should check if things don't work out the way we want them to; and maybe we shouldn't rely on it to begin with if the stakes are too high. But in the absence of further evidence, it is the most likely explanation.
That said, some amount of doubt is in order, particularly if we're talking about simple studies and complex systems. In that situation, we're bound to miss subtleties. And I guess I'm unclear what your overall thesis here is. Is it "we don't know anything until we've studied male left-handed marathon runners that prefer clockwise tracks vs male right-handed marathon runners that prefer clockwise tracks" (really needs an acronym) or is it "if you know you're a marathon runner, you might want to use that bit of information when estimating the health benefits of walking more"?
Part by part.
3.5) Another thing you did with your example is that you moved from the probabilistic approach when studying populations to a deterministic approach (100% of marathoners). Empirically, if something happens 100% of the time is not part of a study because it is already common knowledge. It follows the laws of nature, like the effect of gravity. When you switch to a deterministic approach two things are evident a)The deterministic trick is a straw man fallacy by itself b) The deterministic trick open the door to interpretations like "I did smoke all my life and I am in my 90's, therefore smoking does not kill...," which are correct at the individual experience level ("does not kill ME") and wrong at the population level (smoking increases the rate of mortality).
I hope this helps.
I'm not sure I have either the patience or the words to communicate about this entire matter effectively. A bunch of what you said strikes me as a very different (from mine) conceptualizing of chance and probability, or drastically different words to express that conceptualization, also considering the "english is not my first language" part. It could also be an insufficient understanding of statistics but I'm going to assume it's not, and try and clean up as many possible misunderstandings as I reasonably can. If an innocent bystander (you, dear reader) happens to understand both of us well, and help in untangling wires is appreciated.
No contest. That was exactly my point.
Let me put it like this: You roll a die. It lands on 6. What was the chance of you rolling a 1, assuming a fair die? I hope you agree it's 1/6, and not 0. If you say the chance was 0, because ultimately it ended up on a different number, then I understand why you'd say that the 25% reduction makes no sense on an individual level. But same as a dice roll, whether you die is in the above scenario ultimately random. And by walking more, you can affect the probability of that random event. Therefore, your probability of dying decreased. It is possible that there is no causal link in your case between walking more and dying less, because you're a left handed marathon man. But since we're not observing at that level of detail, and hence have no information how your left handedness interacts with your walking and health, we subsume it as randomness. Consider also that the physics of how the dice rolls is theoretically predictable, so it ultimately is a deterministic process (save for some potential marginal truly random quantum effects), but we don't conceptualize it as such because no one has time to model a dice roll to such a level of detail as to make it deterministic. From that perspective, while there might be a theoretical deterministic chain of events from initial conditions to you dying or not, we talk about risks, because no one resolves or understands reality to such a degree of detail as to make that determinism useful. Hence: We speak in probabilities, and we resolve those probabilities as fine-grainedly as the available information supports. Put very very simply: How do you know the intervention did not save you? If you don't know, you must admit that probability is the right terminology to talk about the effect.
3.5: There is no determinism I invoked. The 100% is a 100% in relative terms, i.e. when 5 marathon runners died in the control, 10 did in the intervention, therefore a 100% relative increase. The absolute increase is 1PP.
To clarify: I'm not denying that effects can vary by subgroups, quite the contrary. It can and will happen. But absent any evidence for which subgroups to look at, amongst the infinitely many different axes you could use to separate, you should assume that any individual subgroup split has no effect, while being mindful that there are some for which that assumption is wrong, but you don't know which ones that are.
I understand. Depending on how exactly you mean that, it's either in conflict with my assertion that you should assume that each split is irrelevant, or it isn't. I guess the essential question then is whether you apply any kind of "presumption of irrelevance", or do some other kind of filtering, and if so, how. I.e., when you read the study in the previous-linked article, do you simply say "I'm different from the aggregate population they studied", or do you say "but I live at 4000m above sea level, therefore I have reason to believe my cardiovascular system reacts differently to exercise than that of the aggregate population they studied"? And how do you decide whether "4000m above sea level" for example is sufficient difference to discount the results? Or that "but I'm left-handed" is insufficient?
I hope my reply is even half as lucid as I think it is.
I do agree, it is hard to communicate. We do agree in many points and we have disagreements in a few.
The rolling a die is a clear example of ecological fallacy. Let's see whether can share some common understanding.
What is going to be the next roll? It can be any from one to six. Individually, I do care that it will be not 1 (dead), it can be any of the others (alive). What will be the next one? It cannot be answered with 5/6 chances of being not one because that was not the question. The next roll will be one or will be not one and, therefore the answer should be "it will be one" or "it will not be one." The question at the individual level can be answered by, as you say, having (almost) complete knowledge of all factors involved in that roll. In that roll, not in a series of rolls from where the uniform, and the 1/6 and 5/6, probabilities were obtained. The answer providing probabilities of the next event works very well rolling the dice many times. It answers a different question. Not my question about the next roll.
Now, if I walk more, will my health be better? According to a study, those who walk more had higher chances of being healthy. But, I am asking about myself because I want to decide whether to walk more or not. Let's say that I was part of the study and I do not have an answer because there is no control group for me. There is a control group for the group I belong to, not for me. Therefore, answering my question about me with the chances of the group is like predicting the next roll based on probabilities obtained with a series of rolls in which I am not interested because I want to know the answer about me.
The only way to be right predicting individual outcomes with group probabilities is predicting multiple individual outcomes, which is not an individual prediction but a group prediction. Some will be correct prediction and some will be wrong predictions and the average likely will be the probabilities of the initial study, the series of rolls.
Poker players play using statistics and they know that if they played multiple times according to those probabilities at the end they will make some money. But they cannot know whether they will make money in each individual play.
The answer about individual prediction cannot come from epidemiological studies but for clinical work. 80% of long term smokers do not die of lung cancer although their risk is increased 30 times in comparison with non-smokers. How can you explain that except by saying that the increased risk is for the population, not for each smoker. Why? The answer is we do not know yet. We pretend we know to make those who smoke a lot to quit or smoke less, to save the life of at least some of those who would have died due to lung cancer.
Thanks.
Yes, I suspect this might be true of lots of drugs? Strong effects for some but no effect for others. Maybe it would be clearer to explain it as people varying rather than an "ecological fallacy" which is rather abstract?
An example might be psychiatrists who will try different drugs until they find one that works.
But I'm not sure what that means other than setting expectations about needing to do experiments? Trying a different drug is an n=1 experiment.
It's expensive, though, and requires persistence. Figuring out cheap tests to see if someone will benefit can improve the odds. It may be difficult to find a test that's as good as actually trying it.
More cynically, though, drug companies benefit from people taking drugs because they might help. Having a good, cheap test to tell you that a drug won't work for you would reduce market size, perhaps drastically, turning it into a specialty drug.
I do agree with you that it might be clearer explaining as variation in response and not with ecological fallacy, which is, as you say, exploited by pharmaceutical companies.
Clinical work (I go to my nurse practitioner) is N=n=1 and my nurse study me repeated times and try what works.
Currently, clinical work is a mix of epidemiology and knowledge about the individual patient. What patients demand is what is going on with me (diagnosis) and what will happen to me (prognosis). What public health institutions want is what is going on and what will happen with populations, groups of people.
Clinical work is, as you say, very expensive. We are very far from individualized medicine.
It's just as much of an error to say "this correlative population study means that X will certainly have Y effect for each individual" as it is to say "the fact that this was a correlative population study means that we can't draw any conclusions at all."
Studies suggest that statins reduce the risk of mortality for those at high risk of heart disease. That doesn't mean that they will work for everyone in that population, but if you're at high risk for heart disease, you should take statins.
This. The ecological fallacy is a thing but OP seems to be over-extending the concept to conclude that it's impossible to ever draw conclusions from studies. You could use their same arguments to conclude that smoking doesn't cause lung cancer (and tobacco companies have used that argument in the past to avoid regulation).
It's like, yeah, I can't guarantee something about an individual based on these studies, but that doesn't mean statistics just cease to apply to them. And as for the only way to draw conclusions being working with a clinician... how do you think the clinician learns how certain things manifest and what to look out for? Unless they have precognition, they're taking some level of statistical risk factors into consideration when they assess your condition because those things can affect how they interpret their observations.
That is wrong, the number needed to treat for statins hoover around 200-300, implying that to benefit one patient, the one that will improve his/her health with statins, 200 or 300 have to be treated and receive no benefit. Physicians know that and prescribe statins because if they do not that one lucky person will never receive the good effect.
We take statins, and every medicine, with a grain of salt.
That seems inherent in "risk of mortality" though? We are doing probability calculations because we don't know what will happen and outcomes vary.
Given that, I don't see anything wrong. It seems good to emphasize our ignorance about what will happen, though.
Yes. With any rate, mortality included.
Probability calculations because at the group level its members vary randomly. Nothing wrong with that.
The fallacy is to assume that the probability of the group is the probability of each individual member of the group. We do not know what will happen at the individual level and we have a good idea of what will happen at the group level.
I do agree. Well said. Association is necessary for causality. Just not enough to assert it.
Added: from a public health point of view, it would be cringe to say "stop smoking because is going to kill some of you, a few... maybe 10-20% of long term smokers" (https://www.sciencedaily.com/releases/2022/04/220411113733.htm). In fact, most smokers do not get lung cancer and, despite that, epidemiologically smoking causes lung cancer.
Drugs are typically approved based on randomized controlled trials. Was that not true for statins?