- Published on
Chi-square p-values are not enough
Most people that need statistics are focused only on the almighty p-value of less than .05. When running Chi-square analyses between a dichotomous categorical predictor and a dichotomous categorical outcome, p-values are not the primary inference that should be interpreted for practical purposes. The lack of precision and accuracy in categorical measures coupled with sampling error makes the p-values yielded from Chi-square analyses virtually worthless in the applied sense.
The correct statistic to run is an unadjusted odds ratio with 95% confidence interval. This is the best measure for interpreting the magnitude of the association between two dichotomous categorical variables collected in a retrospective fashion. Relative risk can be calculated when the association is assessed in a prospective fashion.
The width of the 95% confidence interval and it crossing over 1.0 dictate the significance and precision of the association between the variables. With smaller sample sizes, the 95% confidence interval will be wider and less precise. Larger sample sizes will yield more precise effects.
The correct statistic to run is an unadjusted odds ratio with 95% confidence interval. This is the best measure for interpreting the magnitude of the association between two dichotomous categorical variables collected in a retrospective fashion. Relative risk can be calculated when the association is assessed in a prospective fashion.
The width of the 95% confidence interval and it crossing over 1.0 dictate the significance and precision of the association between the variables. With smaller sample sizes, the 95% confidence interval will be wider and less precise. Larger sample sizes will yield more precise effects.
1 Comments
Great article, Aly! Thank you for pointing out the importance of interpreting odds ratios with 95% confidence intervals when running Chi-square analyses. As you mentioned, p-values are often overemphasized, but the precision and accuracy of categorical measures can be improved by using odds ratios with confidence intervals. This is a crucial aspect of statistical analysis that researchers and practitioners alike should keep in mind.