Galleons, the primary reason I value science over religion is that science pursues the truth, whereas religion does not. And the search for truth is a constantly shifting thing. We build on it, tear it apart, start again. As our technology advances, our insights into the mysteries of the universe deepen. As we grow as a society, it is our science that is slowly unveiling the truth around us.
There is nothing more beautiful than that. Nothing more important.
But even scientists are human, prone to stumbling, mistakes, oversights. There are rules and methods in place to minimize subjectivity in experiments and to make sure all studies and experiments are held to rigorous standards. And yet, the human element slips through.
This becomes particularly problematic when that human element causes a flawed test/experiment that is not caught for years because scientists get lax and don’t follow the rules. If they let the one study stand as fact, without replicating it to test the validity of its assertions, then we have a problem. A big one.
As my hero/idol/personal god Richard Feynman once said:
When I was at Cornell, I often talked to the people in the psychology department. One of the students told me she wanted to do an experiment that went something like this–it had been found by others that under certain circumstances, X, rats did something, A. She was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under circumstances Y and see if they still did A.
I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person–to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know the the real difference was the thing she thought she had under control.
She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happened.
Nowadays, there’s a certain danger of the same thing happening, even in the famous field of physics. I was shocked to hear of an experiment being done at the big accelerator at the National Accelerator Laboratory, where a person used deuterium. In order to compare his heavy hydrogen results to what might happen with light hydrogen, he had to use data from someone else’s experiment on light hydrogen, which was done on different apparatus. When asked why, he said it was because he couldn’t get time on the program (because there’s so little time and it’s such expensive apparatus) to do the experiment with light hydrogen on this apparatus because there wouldn’t be any new result. And so the men in charge of programs at NAL are so anxious for new results, in order to get more money to keep the thing going for public relations purposes, they are destroying–possibly–the value of the experiments themselves, which is the whole purpose of the thing. It is often hard for the experimenters there to complete their work as their scientific integrity demands.
What is true of psychology and physics is true across the board. Sometimes, scientific integrity slips. And this is a major concern.
I bring this up because there is evidence that an extremely well-known study involving fruit flies (people always think about lab rats, but I think we discuss experiments involving fruit flies with much greater frequency) has recently come under fire as being “fatally flawed.” It’s Angus John Bateman’s 1948 study regarding the evolutionary advantage male fruit flies gain from being promiscuous.
It’s kind of a big deal.
See, this study was huge. It has informed and influenced an entire sub-field of evolutionary biology. As Patricia Adair Gowaty, a professor of ecology and evolutionary biology at UCLA, said, “Bateman’s 1948 study is the most-cited experimental paper in sexual selection today because of its conclusions about how the number of mates influences fitness in males and females.”
However, in all the intervening decades, the experiment has never been repeated using the exact methods Bateman used. Until now. And the results are shaking the foundation of this section of evolutionary biology.
So, in case you are unfamiliar with the experiment (or just need a refresher), here’s what it’s all about:
Bateman created isolated little populations of fruit flies in jars, either 5 males and 5 females or 3 males and 3 females. Within their isolation, the fruit flies were free to mate at will. Bateman then studied the progeny that survived to adulthood. However, even within these isolated populations, Bateman needed a reliable way to determine the parents of the grown children.
Enter the defects.
Bateman selected first generation fruit flies with very specific genetic mutations. Squinty eyes. Curled wings. Thick bristles. Without the technology of today that allows us to determine the genetic parentage of the flies, Bateman needed strong, visible, transferable mutations.
So, by placing 6 or 10 flies in these communities with 6 or 10 different, extreme mutations, Bateman could work backward to determine the parentage of the fly children.
Well, some of them, that is. And that’s where the problem lies. See, if you know your genetics, you know those fly babies could display the mutations of both parents, the mutation of only one parent, or no mutations at all. So Bateman could only study the fly children that demonstrated two mutations because they were the only ones that could have both parents reliably identified.
Maybe you see the problem now. The sample was skewed.
Gowaty and her team were the ones who replicated Bateman’s famous experiment, and they found that those double mutation children were the least likely to survive to adulthood (as you are probably aware, mutations often represent a disadvantage to the carrier). So the roughly 25% of the fly progeny exhibiting the double mutations were the most likely to die, shrinking the sample even further. With a sample that small, there’s no way Bateman could accurately equate the number of mates for each adult subject.
But that’s not the only problem with Bateman’s study. Turns out, according to his methodology, more offspring were assigned to fathers than to mothers (leading to the conclusion that male flies were more promiscuous). But all of those fly babies need both a father and a mother.
So, Bateman’s study has some glaring issues. But were the conclusions sound?
Uh… the results are inconclusive, as it turns out.
The markers used to determine parentage (those pesky mutations) actually influence the parameters being measured. Because when the mutated children die off before reaching adulthood (as they are more likely to do), the results become biased. The children are no longer an accurate measurement of the number of mates each adult fly had, because not all of them survived.
“Here was a classic paper that has been read by legions of graduate students, any one of whom is competent enough to see this error,” Gowaty said. “Bateman’s results were believed so wholeheartedly that the paper characterized what is and isn’t worth investigating in the biology of female behavior.”
Let this serve as a lesson to the scientific community. This is a study that should have been retried immediately. Certainly at some point in the last 60+ years, right? We can’t be constrained by strong theories and paradigms. In the world of science, all it takes is one person with a new perspective to come along and shatter these existing limitations, stretching our knowledge and pushing us ever closer to the truth. That is the purpose driving every scientist, that rigorous pursuit of truth.
Here’s hoping that we take away from this revelation a renewed understanding of the need for strong scientific integrity. And here’s also hoping that shattering this foundation allows scientists like Gowaty (who studies female mating habits in various species) a chance to find the truth, unhampered by the constrictions of paradigms set by a flawed study.