Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:
“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”
101 of 103 studies were not dismissed. All systematic reviews classify their source studies based on the quality of the work. Of the 103, two were classed as high quality, 58 as moderate quality and the remaining 43 as low quality. For synthesis, only high and moderate quality studies were drawn on. That’s more than half, not 2%.
So yes, Erin is lying.
You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies for the numbers it used to reach its conclusion. You can’t say she’s lying without that review no more than I can support Erin by reading each study that was dismissed. What I can tell you is that dismissing that many studies is not normal scientific analysis. It reeks of bias.
This is the lie. They didn’t dismiss all but two studies, they actually included 60. More than half of the 103 studies identified for the review.
So yes, Erin, and now yourself, are peddling a lie.
It’s key part of synthesising multiple sources into a meta-analysis. Including poor quality studies dilutes the quality of the overall analysis.
By design, it’s biased towards higher quality research.