Adjusting for multiple comparisons
Correct for increased Type I error rates when testing multiple comparisons concurrently
Researchers often conduct multiple statistical tests concurrently on several different outcomes. Most researchers are also interested in answering more than just one question in their research! However, as the number of hypotheses being tested increases, the odds of committing a Type I error (falsely rejecting the null hypothesis or false-positive) increases drastically for each hypothesis. These increasing rates in Type I error are known as experimentwise error rates.
Experimentwise error rates can be accounted for by adjusting for multiple comparisons. There are three primary tests that are used when adjusting for multiple comparisons and each is employed depending upon the current empirical context: Bonferroni, Tukey's HSD, and Scheffe's test. Adjusting for multiple comparisons means adjusting the level of significance to be more stringent in light of the increased experimentwise error rates. With a more stringent alpha level needed to achieve statistical significance, the chances of committing a Type I error decrease.
Most journals now require adjusting for multiple comparisons to publish research findings. It will show more empirical rigor on a researcher's part to adjust for multiple comparisons the first time around and do things the right way.
Experimentwise error rates can be accounted for by adjusting for multiple comparisons. There are three primary tests that are used when adjusting for multiple comparisons and each is employed depending upon the current empirical context: Bonferroni, Tukey's HSD, and Scheffe's test. Adjusting for multiple comparisons means adjusting the level of significance to be more stringent in light of the increased experimentwise error rates. With a more stringent alpha level needed to achieve statistical significance, the chances of committing a Type I error decrease.
Most journals now require adjusting for multiple comparisons to publish research findings. It will show more empirical rigor on a researcher's part to adjust for multiple comparisons the first time around and do things the right way.
Statistical tests that adjust for multiple comparisons
Hire A Statistician
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D., PStat will provide the following statistical consulting services for undergraduate and graduate students at $100/hour. Secure checkout is available with Stripe, Venmo, Zelle, or PayPal.
- Statistical Analysis
- Research Design
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Survey Design and Psychometrics
