# Random-effects ANOVA

## Test interactions between multiple observations of outcomes

Random-effects ANOVA is used to answer research questions where the variance across observations and within-subjects effects can be assessed across different levels of categorical variables. With random-effects ANOVA, there are demographic, prognostic, and clinical variables that confound and mitigate associations between predictor and outcome variables. Random-effects ANOVA allows you to answer these more complex research questions, and thus, generate evidence that is more indicative of the outcome as it truly exists in the population of interest. The

For example, let's say researchers are interested in the effects of a new therapy for people with social anxiety and the number of sick days they use yearly. A validated measure of social anxiety would be administered to study participants to assess their baseline level of social anxiety and the number of sick days they had utilized at baseline. Then, at six months after participating in the new therapy regimen, a second observation of social anxiety and sick days used would be taken. The same would be repeated at 12 months. Therefore, you can assess how these two "random" effects change and interact with each other across time or within-subjects.

Researchers are essentially answering three research questions with one analysis: Was there a significant main effect across time for the first "random" variable? Was there a significant main effect across time for the second "random" variable? Do the two "random" variables adjust the outcome variable in a significant fashion?

The

**random-effects ANOVA**focuses on how "**random**" observations of an outcome vary across two or more within-subjects variables.For example, let's say researchers are interested in the effects of a new therapy for people with social anxiety and the number of sick days they use yearly. A validated measure of social anxiety would be administered to study participants to assess their baseline level of social anxiety and the number of sick days they had utilized at baseline. Then, at six months after participating in the new therapy regimen, a second observation of social anxiety and sick days used would be taken. The same would be repeated at 12 months. Therefore, you can assess how these two "random" effects change and interact with each other across time or within-subjects.

Researchers are essentially answering three research questions with one analysis: Was there a significant main effect across time for the first "random" variable? Was there a significant main effect across time for the second "random" variable? Do the two "random" variables adjust the outcome variable in a significant fashion?

The

**marginal means and errors**for each level of the interaction should be presented.**Significant main effects**must be further tested in a**post hoc**fashion to assess where amongst the levels of the interaction the significance exists and when there are more than two**"random**" observations of the outcome**.**The figure below depicts the use of random-effects ANOVA. There is only one group being observed. Observations of two difference within-subjects variables are being taken at multiple times moving forward. Random-effects ANOVA yields marginal means and standard errors.

### The steps for conducting a random-effects ANOVA

1. The data is entered in a within-subjects fashion.

2. Click

3. Drag the cursor over the

4. Click

5. Type the name of the first "random" or within-subjects variable into the

6. Type the number of observations of the outcome into the

7. Click the

8. Type of the name of the second "random" or within-subjects variable into the

9. Type the number of observations of the outcome into the

10. Click the

11. Click

12. Click on the first observation of the first "random" or within-subjects variable to highlight it.

13. Click on the

14. Click on the second observation of the first "random" or within-subjects variable to highlight it.

15. Click on the

16. Click on the first observation of the second "random" or within-subjects variable to highlight it.

17. Click on the

18. Click on the second observation of the second "random" or within-subjects variable to highlight it.

19. Click on the

20. Click on the

21. Click on the first "random" or within-subjects variable to highlight it.

22. Click on the

23. Click on the second "random" or within-subjects variable to highlight it.

24. Click on the

25. Click on the

26. Click

27. Click on the

28. In the

29. Click on the

30. Repeat Steps 26 and 27 until all of the "random" or within-subjects variables and the interaction are moved into the

31. Click on the

32. In the

33. Click

34. Click

2. Click

**.**__A__nalyze3. Drag the cursor over the

**drop-down menu.**__G__eneral Linear Model4. Click

**.**__R__epeated-Measures5. Type the name of the first "random" or within-subjects variable into the

**box.**__W__ithin-Subject Factor Name:6. Type the number of observations of the outcome into the

**Number of**box.__L__evels:7. Click the

**button to move the variable into the box.**__A__dd8. Type of the name of the second "random" or within-subjects variable into the

**box.**__W__ithin-Subject Factor Name:9. Type the number of observations of the outcome into the

**Number of**box.__L__evels:10. Click the

**button to move the variable into the box.**__A__dd11. Click

**Define**.12. Click on the first observation of the first "random" or within-subjects variable to highlight it.

13. Click on the

**arrow**to move the variable into the**(1,1) row of the Within-Subjects Variables (Variable 1 name, Variable 2 name): box**.14. Click on the second observation of the first "random" or within-subjects variable to highlight it.

15. Click on the

**arrow**to move the variable into the**(1,2) row of the Within-Subjects Variables (Variable 1 name, Variable 2 name): box**.16. Click on the first observation of the second "random" or within-subjects variable to highlight it.

17. Click on the

**arrow**to move the variable into the**(2,1) row of the Within-Subjects Variables (Variable 1 name, Variable 2 name): box**.18. Click on the second observation of the second "random" or within-subjects variable to highlight it.

19. Click on the

**arrow**to move the variable into the**(2,2) row of the Within-Subjects Variables (Variable 1 name, Variable 2 name): box**.20. Click on the

**Plo**button.__t__s21. Click on the first "random" or within-subjects variable to highlight it.

22. Click on the

**arrow**to move the variable into the**box.**__S__eparate Lines:23. Click on the second "random" or within-subjects variable to highlight it.

24. Click on the

**arrow**to move the variable into the**box.**__H__orizontal Axis:25. Click on the

**button.**__A__dd26. Click

**Continue**.27. Click on the

**button.**__O__ptions28. In the

**table, click on the first "random" or within-subjects variable to highlight it.**__F__actor(s) and Factor Interactions:29. Click on the

**arrow**to move the variable into the**Display**box.__M__eans for:30. Repeat Steps 26 and 27 until all of the "random" or within-subjects variables and the interaction are moved into the

**Display Means for:**box.31. Click on the

**C**box to select it.__o__mpare main effects32. In the

**Display**table, click on the**,**__D__escriptive statistics**, and**__E__stimates of effect size**Observed power**boxes to select them.33. Click

**Continue**.34. Click

**OK**.### The steps for interpreting the SPSS output for a random-effects ANOVA

1. Scroll down to the

2. Look at the value for the first "random" or within-subjects variable.

If the

If the

3. Look at the

If the

If the

4. Look at the

If the

If the

5. If researchers have a significant main or interaction effect, scroll down to the

6. For each "random" level or within-subjects variable, look in the

7. For each "random" level or within-subjects variable, look in the

If the

If the

8. Scroll down to the table with the

9. These are the

10. The graph provides a visual aid at understanding the main and interaction effects.

**Tests of Within-Subjects Effects**table and look in the**Sig.**column. These are the*p*-values that are interpreted. Interpret the**Greenhouse-Geisser**corrected value.2. Look at the value for the first "random" or within-subjects variable.

If the

*p*-value is**LESS THAN .05**, then researchers have evidence of a statistically significant change across time or within-subjects for that variable.If the

*p*-value is**MORE THAN .05**, then researchers do**NOT**have evidence of a statistically significant change across time or within-subjects.3. Look at the

*p*-value for the second "random" or within-subjects variable.If the

*p*-value is**LESS THAN .05**, then researchers have evidence of a statistically significant change across time or within-subjects for that variable.If the

*p*-value is**MORE THAN .05**, then researchers do**NOT**have evidence of a statistically significant change across time or within-subjects for that variable.4. Look at the

*p*-value for the**first*second**"random" effects interaction.If the

*p*-value is**LESS THAN .05**, then researchers have evidence of a statistically significant change across time or within-subjects for the interaction.If the

*p*-value is**MORE THAN .05**, then researchers do**NOT**have evidence of a statistically significant change across time or within-subjects for the interaction.5. If researchers have a significant main or interaction effect, scroll down to the

**Estimated Marginal Means**section of the output.6. For each "random" level or within-subjects variable, look in the

**Estimates**table for the means and standard errors for each observation.7. For each "random" level or within-subjects variable, look in the

**Pairwise Comparisons**table under the**Sig.**column. These are the*p*-values associated with the post hoc comparisons.If the

*p*-value is**LESS THAN .05**, then researchers have evidence of a statistically significant change across time or within-subjects for that variable.If the

*p*-value is**MORE THAN .05**, then researchers do**NOT**have evidence of a statistically significant change across time or within-subjects for that variable.8. Scroll down to the table with the

**first*second**"random" variables names as the title.9. These are the

**marginal means and standard deviations**for each level of the interaction between the "random" variables.10. The graph provides a visual aid at understanding the main and interaction effects.

Click on the

**Download Database**and**Download Data Dictionary**buttons for a configured database and data dictionary for random-effects ANOVA.**Click on the****Validation of Statistical Findings**button to learn more about bootstrap, split-group, and jack-knife validation methods.## Hire A Statistician

**DO YOU NEED TO HIRE A STATISTICIAN?**

Eric Heidel, Ph.D., PStat** **will provide the following statistical consulting services for undergraduate and graduate students at $100/hour. Secure checkout is available with Stripe, Venmo, Zelle, or PayPal.

- Statistical Analysis
- Research Design
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Survey Design and Psychometrics