Publishing only significant research findings
The collective unconscious and statistics
Admit it, it just feels better when a statistically significant difference or treatment effect is found! Promotion, tenure, benefits, and perks...all relying on the one p-value being below .05! Statistics has done something for you when have found statistical significance!
Truth be told, this line of thinking has lead to a gross overestimation OR underestimation of important treatment effects in the clinical literature. Publication bias is a rampant, unconscious, and deleterious phenomenon within science. But, so long as human beings with presuppositions, biases, and knowledge gaps conduct research and statistics AND so long as human beings are responsible for peer-review and gate-keeping of the literature, publication bias will continue to exist. This means that important and potentially life-saving or cost-saving treatments will not be represented in the clinical literature, SIMPLY BECAUSE STATISTICAL SIGNIFICANCE WAS NOT ACHIEVED.
What can be done? I proffer the following:
1. I think that math, science, and statistics need a complete "makeover" in the collective unconscious. These things are very cool and it is completely awesome to be a nerd and work hard towards mastering content areas within these fields. College degrees in these fields lead to job security and better pay.
In my experience, few people recall their experiences with statistics with much zeal. People have an automatic recoil towards statistics. This MUST change. If statistics and hypothesis testing are going to be the methods by which we conduct, communicate, and interpret research findings, then a drastic change in the collective orientation towards statistics must occur.
2. In tune with #1, statistical scientists and educators must do a better job of teaching the lexicon or language of our mathematical science to the general population. Skewness, kurtosis, effect size, sample size, statistical power, confidence interval, probability, reliability, validity, precision, accuracy, sampling error, normality, homogeneity of variance, sphericity, standard deviation, variance, covariance, confounding, hypothesis testing, reject, do not reject, Type I error, Type II error...WHAT DO ALL OF THESE WORDS MEAN???
Most people do not use these words everyday. People experience cognitive dissonance when knowledge and sensory gaps occur. Educators need to take a deductive approach towards imparting the language and "meaning" within content areas such as 1) hypothesis testing, 2) measurement, 3) statistical power, and 4) statistics. Without a basic working knowledge and understanding of these critical terms in applied statistics, the collective unconscious will continue to recoil at the very sight, sound, or presence of statistics, and even some statisticians.
3. Masters and doctoral level researchers and clinicians need to possess the knowledge and experience to conduct applied statistics in the correct fashion. I could really get on my "soapbox" here but I've almost taken too much of your time. Simply put, seek out assistance before publishing data that does not meet statistical assumptions.
In conclusion, I hope you can appreciate my candor in this regard. Publication bias is a REAL phenomenon and it has DRASTIC implications on clinical treatment. You want your clinician to be informed by unbiased clinical evidence. However, that is probably not the case. Let's work towards changing this scary truth!
Truth be told, this line of thinking has lead to a gross overestimation OR underestimation of important treatment effects in the clinical literature. Publication bias is a rampant, unconscious, and deleterious phenomenon within science. But, so long as human beings with presuppositions, biases, and knowledge gaps conduct research and statistics AND so long as human beings are responsible for peer-review and gate-keeping of the literature, publication bias will continue to exist. This means that important and potentially life-saving or cost-saving treatments will not be represented in the clinical literature, SIMPLY BECAUSE STATISTICAL SIGNIFICANCE WAS NOT ACHIEVED.
What can be done? I proffer the following:
1. I think that math, science, and statistics need a complete "makeover" in the collective unconscious. These things are very cool and it is completely awesome to be a nerd and work hard towards mastering content areas within these fields. College degrees in these fields lead to job security and better pay.
In my experience, few people recall their experiences with statistics with much zeal. People have an automatic recoil towards statistics. This MUST change. If statistics and hypothesis testing are going to be the methods by which we conduct, communicate, and interpret research findings, then a drastic change in the collective orientation towards statistics must occur.
2. In tune with #1, statistical scientists and educators must do a better job of teaching the lexicon or language of our mathematical science to the general population. Skewness, kurtosis, effect size, sample size, statistical power, confidence interval, probability, reliability, validity, precision, accuracy, sampling error, normality, homogeneity of variance, sphericity, standard deviation, variance, covariance, confounding, hypothesis testing, reject, do not reject, Type I error, Type II error...WHAT DO ALL OF THESE WORDS MEAN???
Most people do not use these words everyday. People experience cognitive dissonance when knowledge and sensory gaps occur. Educators need to take a deductive approach towards imparting the language and "meaning" within content areas such as 1) hypothesis testing, 2) measurement, 3) statistical power, and 4) statistics. Without a basic working knowledge and understanding of these critical terms in applied statistics, the collective unconscious will continue to recoil at the very sight, sound, or presence of statistics, and even some statisticians.
3. Masters and doctoral level researchers and clinicians need to possess the knowledge and experience to conduct applied statistics in the correct fashion. I could really get on my "soapbox" here but I've almost taken too much of your time. Simply put, seek out assistance before publishing data that does not meet statistical assumptions.
In conclusion, I hope you can appreciate my candor in this regard. Publication bias is a REAL phenomenon and it has DRASTIC implications on clinical treatment. You want your clinician to be informed by unbiased clinical evidence. However, that is probably not the case. Let's work towards changing this scary truth!
