MANCOVA
Account for increased Type I error when comparing independent groups on multiple adjusted outcomes
MANCOVA is often employed to account for what is called "increased experimentwise error rates" when testing multiple hypotheses concurrently and adjusting several outcome values at the same time with a covariate. Essentially, with each mutually exclusive analysis between the categorical predictor variable and the adjusted outcome, the chances of committing a Type I error increase substantially. If the statistical assumptions of a MANCOVA can be met, it is a much more powerful inferential statistic that can yield both main and interactional effects while controlling for a covariate.
The figure below depicts the use of MANCOVA. The differences between independent groups are being adjusted for by a covariate when assessing multiple continuous outcomes.
The steps for conducting MANCOVA in SPSS
1. The data is entered in a between-subjects fashion.
2. Click Analyze.
3. Drag the cursor over the General Linear Model drop-down menu.
4. Click Multivariate.
5. Click on the first continuous outcome variable to highlight it.
6. Click on the arrow to move the variable into the Dependent Variables: box.
7. Repeat Steps 5 and 6 until all of the continuous outcome variables are in the Dependent Variables: box.
8. Click on the categorical predictor variable that represents independent groups or levels.
9. Click on the arrow to move the variable into the Fixed Factor(s): box.
10. Click on the continuous covariate variable to highlight it.
11. Click on the arrow to move the variable into the Covariate(s): box.
12. Click on the Options button.
13. Click on the categorical predictor variable in the Factor(s) and Factor Interactions: box to highlight it.
14. Click on the arrow to move the variable into the Display Means for: box.
15. Click on the Compare main effects box to select it.
16. In the Display table, click on the Descriptive statistics, Estimates of effect size, Observed power, and Homogeneity tests boxes to select them.
17. Click Continue.
18. Click OK.
2. Click Analyze.
3. Drag the cursor over the General Linear Model drop-down menu.
4. Click Multivariate.
5. Click on the first continuous outcome variable to highlight it.
6. Click on the arrow to move the variable into the Dependent Variables: box.
7. Repeat Steps 5 and 6 until all of the continuous outcome variables are in the Dependent Variables: box.
8. Click on the categorical predictor variable that represents independent groups or levels.
9. Click on the arrow to move the variable into the Fixed Factor(s): box.
10. Click on the continuous covariate variable to highlight it.
11. Click on the arrow to move the variable into the Covariate(s): box.
12. Click on the Options button.
13. Click on the categorical predictor variable in the Factor(s) and Factor Interactions: box to highlight it.
14. Click on the arrow to move the variable into the Display Means for: box.
15. Click on the Compare main effects box to select it.
16. In the Display table, click on the Descriptive statistics, Estimates of effect size, Observed power, and Homogeneity tests boxes to select them.
17. Click Continue.
18. Click OK.
The steps for interpreting the SPSS output for MANCOVA
1. Look in the Box's Test of Equality of Covariance Matrices, in the Sig. row. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have violated the assumption of homogeneity of covariance and should not interpret the outputs further.
If it is MORE THAN .05, then researchers can continue with the analysis and have met the assumption of homogeneity of covariance.
2. Look in the Levene's Test of Equality of Error Variances table, in the Sig. column. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have violated the assumption of homogeneity of variance and should not interpret the outputs further.
If it is MORE THAN .05, then researchers can continue with the analysis and have met the assumption of homogeneity of variance.
3. Look in the Multivariate Tests table, under the Sig. column, for the row that has the continuous covariate variable's name and Pillai's Trace. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have evidence that the covariate adjusts values of the outcome.
If it is MORE THAN .05, then Researchers do not have evidence that the covariate adjusts the outcome. No further interpretation is needed. Report the p-value.
4. Look in the Multivariate Tests table, under the Sig. column, for the row that has the categorical predictor variable's name and Pillai's Trace. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have evidence of a significant main effect when controlling for the covariate.
If it is MORE THAN .05, then researchers do not have evidence of a significant effect when controlling for the covariate. No further interpretation is needed. Report the p-value.
5. Look in the Levene's Test of Equality of Error Variances table, under the Sig. column, for each outcome variable that has a p-value testing the assumption of homogeneity of variance.
If it is LESS THAN .05, then researchers have violated the assumption and should not continue with the analysis.
If it is MORE THAN .05, then researchers can continue with the analysis and interpretation.
6. Look in the Tests of Between-Subjects Effects table, under the Sig. column, for the row denoting the continuous covariate variable. These are the p-values that are interpreted for each individual covariate. These should be interpreted only if there is a significant main effect for the covariate in the Multivariate Tests table.
If a p-value for one of the outcome variables is LESS THAN .05, then there was a significant main effect among the independent groups or levels for the covariate. If researchers find this significant main effect, they need to run post hoc analyses to explain the effect.
If a p-value for one of the outcome variables is MORE THAN .05, then there was NOT significant main effect among the independent groups or levels for the covariate.
7. Look in the Tests of Between-Subjects Effects table, under the Sig. column, for the row denoting the categorical predictor variable. These are the p-values that are interpreted for each individual outcome variable. These should be interpreted if there is a significant main effect in the Multivariate Tests table.
If a p-value for one of the outcome variables is LESS THAN .05, then there was a significant main effect among the independent groups or levels of that outcome. If researchers find this significant main effect, they need to run post hoc analyses to explain the effect.
If a p-value for one of the outcome variables is MORE THAN .05, then there was NOT significant main effect amongst the independent groups or levels of that outcome.
8. Scroll down to the Estimated Marginal Means section of the output. Look in the Estimates table. These are the adjusted means and standard errors of the outcome for each group or level of the categorical variable.
9. Look in the Pairwise Comparisons table, under the Sig. column. These are the post hoc p-values that are interpreted.
If a p-value is LESS THAN .05, then there was a significant difference in the adjusted means between the independent groups or levels of the categorical predictor variable on the outcome.
If a p-value is MORE THAN .05, then there was NOT a significant difference in the adjusted means between the independent groups or levels of the categorical predictor variable on the outcome.
If it is LESS THAN .05, then researchers have violated the assumption of homogeneity of covariance and should not interpret the outputs further.
If it is MORE THAN .05, then researchers can continue with the analysis and have met the assumption of homogeneity of covariance.
2. Look in the Levene's Test of Equality of Error Variances table, in the Sig. column. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have violated the assumption of homogeneity of variance and should not interpret the outputs further.
If it is MORE THAN .05, then researchers can continue with the analysis and have met the assumption of homogeneity of variance.
3. Look in the Multivariate Tests table, under the Sig. column, for the row that has the continuous covariate variable's name and Pillai's Trace. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have evidence that the covariate adjusts values of the outcome.
If it is MORE THAN .05, then Researchers do not have evidence that the covariate adjusts the outcome. No further interpretation is needed. Report the p-value.
4. Look in the Multivariate Tests table, under the Sig. column, for the row that has the categorical predictor variable's name and Pillai's Trace. This is the p-value that is interpreted.
If it is LESS THAN .05, then researchers have evidence of a significant main effect when controlling for the covariate.
If it is MORE THAN .05, then researchers do not have evidence of a significant effect when controlling for the covariate. No further interpretation is needed. Report the p-value.
5. Look in the Levene's Test of Equality of Error Variances table, under the Sig. column, for each outcome variable that has a p-value testing the assumption of homogeneity of variance.
If it is LESS THAN .05, then researchers have violated the assumption and should not continue with the analysis.
If it is MORE THAN .05, then researchers can continue with the analysis and interpretation.
6. Look in the Tests of Between-Subjects Effects table, under the Sig. column, for the row denoting the continuous covariate variable. These are the p-values that are interpreted for each individual covariate. These should be interpreted only if there is a significant main effect for the covariate in the Multivariate Tests table.
If a p-value for one of the outcome variables is LESS THAN .05, then there was a significant main effect among the independent groups or levels for the covariate. If researchers find this significant main effect, they need to run post hoc analyses to explain the effect.
If a p-value for one of the outcome variables is MORE THAN .05, then there was NOT significant main effect among the independent groups or levels for the covariate.
7. Look in the Tests of Between-Subjects Effects table, under the Sig. column, for the row denoting the categorical predictor variable. These are the p-values that are interpreted for each individual outcome variable. These should be interpreted if there is a significant main effect in the Multivariate Tests table.
If a p-value for one of the outcome variables is LESS THAN .05, then there was a significant main effect among the independent groups or levels of that outcome. If researchers find this significant main effect, they need to run post hoc analyses to explain the effect.
If a p-value for one of the outcome variables is MORE THAN .05, then there was NOT significant main effect amongst the independent groups or levels of that outcome.
8. Scroll down to the Estimated Marginal Means section of the output. Look in the Estimates table. These are the adjusted means and standard errors of the outcome for each group or level of the categorical variable.
9. Look in the Pairwise Comparisons table, under the Sig. column. These are the post hoc p-values that are interpreted.
If a p-value is LESS THAN .05, then there was a significant difference in the adjusted means between the independent groups or levels of the categorical predictor variable on the outcome.
If a p-value is MORE THAN .05, then there was NOT a significant difference in the adjusted means between the independent groups or levels of the categorical predictor variable on the outcome.
Click on the Download Database and Download Data Dictionary buttons for a configured database and data dictionary for MANCOVA. Click on the Validation of Statistical Findings button to learn more about bootstrap, split-group, and jack-knife validation methods.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics