I wrote a few weeks back about my disgust that Scientific American would post articles which clearly do not value diversity. Given that Nature has also been called on the carpet for this, I find it refreshing and ironic that the Nature Publishing Group (parent to both of these publications) would now sponsor a diversity week this week. Whatever their motivations, I am please to see not only a devotion to representing diversity, but scientific data backing the importance many of us have recognized for years.
Anyway, they may or may not choose to publish this letter, but the feedback I got and revisions made for clarity improved my message. And I really shouldn’t say just mine, as I received a wonderful outpouring of support to write, edit, and submit this letter. The encouragement I got to challenge a senior colleague for all that is right socially and scientifically was enormous. This is a big thank you to those who have supported me then, historically, and in the future. I appreciate all that you do, and the effort you give me I will be sure to spread to others.
We’ll see how this goes…. and I hope well!
Dear Editors of Scientific American,
I am writing in concern about an article that recently appeared in your magazine. A senior research scientist writing for you at Scientific American chose to tackle, rather adamantly oppose, the inclusion of females in NIH studies. To date, many experiments use males exclusively in research on drug efficacy, and this author proclaims that the inclusion of females is, for “statistical reasons” (a dubious point I will address later), too “expensive” and a “waste of resources” to conduct. For reference, here is the link to the article: http://www.scientificamerican.com/article/testing-males-and-females-in-every-medical-experiment-is-a-bad-idea/?WT.mc_id=SA_Facebook
Women comprise the majority of humans on this Earth, and although the author ignores this point (and perhaps he should, as majority and minority need recognition), I do agree with the opening sentences of his article: “Sex differences lie at the core of biology. They are the driving force of evolution, and in many cases they are fundamental in health and medicine. The study of sex differences is important work, and more of it should be done.”
However, the basis of this article advocates that broadly incorporating the use of males and females in health studies “costs money and requires a duplication of time and effort”. Besides the ethical concerns I have about the reluctance to represent equality due to concerns of effort and money, I am concerned about what this viewpoint suggests to both science and society. For this reason, I would like to offer an analysis of the argument as it is presented in the article to display not only the apparent faulty scientific reasoning, but also the weak factual basis of the article.
To begin, studies that include only one sex eliminate the opportunity to learn whether there are potential benefits or serious adverse effects to the excluded sex which will receive the drug. When you treat women as small men– as has been done historically, you ignore the genetic and hormonal differences that can alter drug function. For example, in mice, the typical trial animal, progesterone expression modulates inhibitory effects on dendritic cells, and ovarian hormones modulate both the brain and behavior in response to cannabis. Just earlier this year, researchers discovered that male and female humans react differently to Ambien. The most popular sleep drug on the market was found to be metabolized differently between males and females while already out on the market for 20 years. Because women metabolize Ambien more slowly, the drug remained in their system longer. Effectively, women were prescribed about twice the dosage they actually needed, and the side effects of impaired driving were so large in women that it prompted FDA investigation. When the FDA investigated the original trials, which were indeed conducted on both sexes, they found that females metabolized the drug at a significantly slower rate. But, unfortunately, the original discovery of a 45% higher level of the drug found in females was “rationalized away” when the drug proceeded to market (read more about it and other drugs in detail here). Ambien, unfortunately, is currently one of the only prescription drugs in the country with a different suggested dose for men and women, but it’s far from the only drug with sex-specific effects. This is also not just true for drug trials. A recent study showed just 3% of trials to consider surgical effects in both sexes, leaving a whopping 80% to study only the effects in males.
The next issue I have is purely a numbers one. Specifically, the faulty statistics just don’t add up. Although the author attempts to use statistics to support their argument, there is an ignorance from oversimplification that I can’t get behind. The author asserts that adding females makes the variance so great that the combined, larger variance of including males and females will mask the drug’s effectiveness for both sexes. If the drug is intended to be used on both sexes uniformly, it may or may not be effective in either sex, or overall. If there is an effect per sex, it will show up when you have a model that includes sex as a factor, and will remain otherwise missed when you lump sexes together. This detail is important, as your models accounts for the fact that the underlying variances are likely different between the males and females in both control and experimental groups, and that variances are smaller within groups than between groups. For example, the general height of males will vary more closely around the mean height of males (i.e. give a smaller variance) compared than the mean height of males and females. If you include sex as a factor and choose the correct statistical model which compares within and between sexes (multiple comparisons), you can disentangle these effects and find whether the drug overall is effective. Yes, the sample sizes (N; here, the number of animals) will need to increase to account for the degrees of freedom lost by this type of analysis, but not to the earth-shattering, economically-wasteful level the author purports. Here, the factor of treatment being added to sex uses only 2 additional degrees of freedom (when df= N – K – 1), where N = # subjects and K = the number of independent variables. Here, K = 3 in this case because you would add the second factor (sex) as well as the variable for the interaction effect of sex * treatment (i.e., this lets you test if the effects are experienced in different directions by sex, say, if it’s good for males and bad for females, for example).
Now, you may choose a more complicated experimental design because you have evidence to suggest the nature of the effect is complicated (e.g., studies performed over an extended period of time or those in which the effects are not independent, i.e. hormonal experiments where one sex mimics another), which could necessitate ‘using up’ more degrees of freedom. However, if most of the effect is shared between the sexes, a two-fold increase in sample size is not necessary. If the differences between sexes are commonly found to be as trivial as the author claims, the majority of models will be quite simple. Formulae exist to determine, based on your experimental design, what the effective sample sizes need to be based on the effect sizes of the “drug” as well as sex and any interactions. Thus, statistics can provide you more information with only slightly more animals in a single study, versus repeating the study in females (again requiring as large an N as the first study) to test for similar effects. The author’s claim of doubled input simply doesn’t match statistical reality, and the suggestion of later investigating effects in females may actually cost more– in terms of time and money– to conduct.
Keep in mind, N values in mice and other species can be kept quite low (often arguably too low) in medical studies regardless, and therefore the power could be called into question whether or not females are considered. You would need more animals (not double the number) to conduct the research if you include both sexes, but if you set up your experiment in a statistically-informed way, if no effect or trend of sex is found, it is perfectly valid to report no effect of sex and then combine experimental groups and test for an overall effect. Because the author claims effects will not be found often, the combination of males and females after finding no effect is yet another way that including females and males opens the door for more discoveries using roughly the same resources. The shoddy work based on low samples sizes is atrocious as-is, so a better understanding of statistics would not only bolster investing in better, more inclusive science, but also boost the validity of findings regardless. Being cheap about science undermines what we can discover, and the power of what our discoveries broadly mean.
If both females and males are the targets of these drugs, specifically avoiding potential benefits or risks to one sex is costly in itself– it ignores that over half of the population exists and has different health outcomes under certain drug regimes. Aside from the implications of administering drugs for which the effects in the majority of the population are unknown, and the clear errors in statistical thinking, I am also disturbed that this effort by the NIH to address sex-differences in research is pitched as political for 2 reasons:
- The author hasn’t done their homework. Work to include females in trials has been on the agenda of the NSF and NIH as a part of the annual Gender Summit for *years*. Specifically, in Nov. 2013 at the last summit held in DC (of which Janine Clayton, mentioned in the article is *listed as an Organizing Convener*), specific sessions included the application, or lack thereof, of gender in studies within STEM and STEM education. Specifically, an invited panel included experts in public health (Sabra Klein), education and policy (Brian Nosek) calling for representation in studies and with years of DATA to support that the inclusion of males and females has positive benefits within both research and policy. So, no, this didn’t start this January of this year, and it’s data-driven, not merely political as the author appears to assert (although the repeated lowercase use of nih vs. NIH in the article, is).
- The author, ostensibly a scientist who uses data to make decisions, could make an argument with economics as the driving force behind his objection, but he doesn’t. The author also could argue factors like how males and females are housed, among other things, could have confounding effects unless more rigorously controlled; I’d accept that this is a concern, but rigor should be expected in science, and so we’d reach agreement there— if only the point had been made. But, again, it is not about rigor, as we already saw in the statistics example. The author simply says it costs more, while lacking data or even rough estimates based on data on the specific costs and scope to the research, and the projected (by the author’s estimates, lesser) costs to society for not supporting sex-balanced research. In addition, the author’s argument also ignores the cost and societal implications of minimally 2 years delay and non-repeat funding for projects which we would, by the author’s suggestion, later like to investigate the implications of sex. By this same token, knowing that there are species-specific differences in drug responses (part of the reason that so many potential drug fail before reaching the clinical test phase), it is not valid, by the author’s suggestion, to include females only as an afterthought to an experiment conducted on males. The inclusion of females at all levels will help us better understand effects, risks, and applicability of treatments. Furthermore, inclusion from the start, versus an add-in-later attitude will speed the drugs to market that are actually shown to have a positive effect. To use costs of including females as a scapegoat to denounce larger funding issues science is deplorable, and the only message it sends is that females are secondary in science, even in the fight for society’s access to effective medicine.
Overall, I recognize there may be costs to incorporating females, but including females and males should have been done from the beginning. To say it is costly now to include them is akin to saying the right for women to vote is bad, for it will to clog our ballot boxes, waste paper, and eat up time at the polls. Hiding behind the convenience of male subjects and weak arguments of thriftiness to justify scientifically biased, unethical treatment of others is not ok. I am embarrassed by the work you chose to publish, and I am not, nor am I ever, inclined to support a viewpoint that is not based on current research, shows a poor grasp of the issue and its history, and does not begin to support itself with any type of data. Frankly, it appears lazy, and that’s not what I’ve come to expect from people supposedly qualified to comment on an issue.
Scientists have an ethical responsibility to represent science and its issues accurately to the public. I hope to see Scientific American accept this same responsibility and maintain high standards in how it represents science and its impacts on society.
Emily G. Weigel