Intraclass correlation coefficient
Assess inter-rater reliability of ratings made at a continuous level
The Intraclass correlation correlation (ICC) is used to assess agreement when there are two or more independent raters and the outcome is measured at a continuous level. Raters should be independent, but should also be trained in the operational definition and identification of the construct. Intraclass correlation coefficient is a more powerful reliability statistic in comparison to Kappa due to the use of continuous measurement.
The steps for conducting an Intraclass Correlation Coefficient (ICC) in SPSS
1. The data is entered in a within-subjects fashion.
2. Click Analyze.
3. Drag the cursor over the Scale drop-down menu.
4. Click Reliability Analysis.
5. Click on the first rater's set of observations to highlight the variable.
6. Click on the arrow button to move the variable into the Items: box.
7. Repeat steps 5 and 6 until all the raters' observations are in the Items: box.
8. Click on the Statistics button.
9. Click on the Intraclass correlation coefficient box to select it.
10. Click on the Model: drop down menu, there are three options.
If researchers do NOT have the same raters rating every observation, use the One-Way Random model.
If researchers have the same raters rating every observation and they are using a sample of raters, use the Two-Way Random model.
If researchers have the same raters rating every observation and they are using the only valid raters (population), use the Two-Way Mixed model.
11. Click Continue.
12. Click OK.
2. Click Analyze.
3. Drag the cursor over the Scale drop-down menu.
4. Click Reliability Analysis.
5. Click on the first rater's set of observations to highlight the variable.
6. Click on the arrow button to move the variable into the Items: box.
7. Repeat steps 5 and 6 until all the raters' observations are in the Items: box.
8. Click on the Statistics button.
9. Click on the Intraclass correlation coefficient box to select it.
10. Click on the Model: drop down menu, there are three options.
If researchers do NOT have the same raters rating every observation, use the One-Way Random model.
If researchers have the same raters rating every observation and they are using a sample of raters, use the Two-Way Random model.
If researchers have the same raters rating every observation and they are using the only valid raters (population), use the Two-Way Mixed model.
11. Click Continue.
12. Click OK.
The steps for interpreting the SPSS output for the ICC
1. Look in the Intraclass Correlation Coefficient table, under the Intraclass Correlation column.
The Single Measures value is used when an individual rating is the level of observation in the outcome.
The Average Measures value is used when the average of several ratings is the level of observation in the outcome.
The Single Measures value is used when an individual rating is the level of observation in the outcome.
The Average Measures value is used when the average of several ratings is the level of observation in the outcome.
Click on the Validity button to continue.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics