Adjusting for multiple comparisons

Researchers often conduct multiple statistical tests concurrently on several different outcomes. Most researchers are also interested in answering more than just one question in their research! However, as the number of hypotheses being tested increases, the odds of committing a Type I error (falsely rejecting the null hypothesis or false-positive) increases drastically for each hypothesis. These increasing rates in Type I error are known as experimentwise error rates. 

Experimentwise error rates can be accounted for by adjusting for multiple comparisons. There are three primary tests that are used when adjusting for multiple comparisons and each is employed depending upon the current empirical context: Bonferroni, Tukey's HSD, and Scheffe's test. Adjusting for multiple comparisons means adjusting the level of significance to be more stringent in light of the increased experimentwise error rates. With a more stringent alpha level needed to achieve statistical significance, the chances of committing a Type I error decrease.

Most journals now require adjusting for multiple comparisons to publish research findings. It will show more empirical rigor on a researcher's part to adjust for multiple comparisons the first time around and do things the right way.