6 votes

Why experiments matter and why we hate them

3 comments

  1. onyxleopard
    Link
    I think the intuitive objections to experimenting on these sorts of variables is an interesting sort of fallacy. I think we intuit that such an experiment will conclude that a certain policy is...

    I think the intuitive objections to experimenting on these sorts of variables is an interesting sort of fallacy. I think we intuit that such an experiment will conclude that a certain policy is superior to the alternatives, and as such, we intuit that those subjects in the experiment who will have been subjected to policies that were found to be inferior by the experiment are now irrevocably disadvantaged, even if informed consent was given. Since experiments are not always conclusive, however, it’s not necessary that such conclusions will be drawn. And, even if an experiment does produce a clear conclusion on the best policy within the scope of the experiment, that doesn’t preclude further experiments from finding even better policies in the future. It’s a strange bias that we have, but it is not uncommon, that we believe that equality is worth preserving in the face of unknowns. The whole reason that science is effective at making progress is because scientists do not accept a lack of evidence as a justification for a policy. Facing a lack of evidence, a scientist is motivated to conduct an experiment, rather than choose a policy at random. The difficulty lies in that it is often difficult to design experiments that can conclusively determine whether a policy is better than the random policy, esp. when experiments involve human subjects.

    The ethical issue of not actively causing harm, such as the study of children lured into refrigerators without informed consent, is an interesting question. I think that the data collected in that study could probably have been collected, albeit more expensively, by setting up recording equipment in refrigerators in the real world and waiting for the experimental conditions to manifest by chance. It’s true that the data may be messier, less controlled, and take far, far longer to collect. But avoiding immoral experimental methods is likely to incur costs, no matter how you slice it. I think ethical scientists would agree that we could learn a lot of things more quickly if we conducted experiments involving human subjects ignoring precautions such as informed consent, esp. experiments where the potential for harm is known in advance. But, ethical scientists don’t perform such experiments because the potential for harm due to unknowns is different than the guarantee (or high likelihood) of harm even if it is experimentally utile. Once scientists experimentally determine a policy is harmful, an experiment may be ended early due to unforeseen complications, and this is still worthwhile. Learning policies to avoid may not be as exciting as determining optimal policies, but it is still good knowledge to have.

    2 votes
  2. jgb
    Link
    Good article. We have to be utilitarian when it comes to randomized trials - if the knowledge gained from a trial could improve outcomes for people across the world for decades or even centuries...

    Good article. We have to be utilitarian when it comes to randomized trials - if the knowledge gained from a trial could improve outcomes for people across the world for decades or even centuries to come, then a not insubstantial amount of risk to the participants is justifiable, as ethically galling as that may seem

    1 vote
  3. skybrian
    Link
    From the article:

    From the article:

    It is unclear why we have this aversion to randomising between two unobjectionable alternatives. The most straightforward explanation is that people either object to the idea of being arbitrarily manipulated, or they are unnerved by the realisation that the clinical director doesn’t know what he or she is doing.

    But while understandable, these are not good arguments against experimentation. If decision makers are fallible — which they are — then randomised trials are a solution to, not a symptom of, that problem. So researchers should work hard to demonstrate the trustworthiness of their experiments. Securing real consent is ethically invaluable, but it is also good public relations.