Statistical Consultation Line: (865) 742-7731
  Statistician For Hire
  • Store

Kappa

A very conservative measure of inter-rater reliability

The Kappa statistic is utilized to generate this estimate of reliability between two raters on a categorical or ordinal outcome. Significant Kappa statistics are harder to find as the number of ratings, number of raters, and number of potential responses increases.

The steps for conducting a Kappa statistic in SPSS

1. The data is entered in a between-subjects fashion.

2. Click Analyze.

3. Drag the cursor over the Descriptives drop-down menu.

4. Click on Crosstabs.

5. Click on the first rater's observations of the outcome to highlight it.

6. Click on the arrow to move the variable into the Row(s): box.

7. Click on the second rater's observations of the outcome to highlight it.

8. Click on the arrow to move the variable into the Column(s): box.

9. Click on the Statistics button.

10. Click on the Kappa box to select it.

11. Click Continue.

12. Click OK.

The steps for interpreting the SPSS output for the Kappa statistic

1. Look at the Symmetric Measures table, under the Approx. Sig. column. This is the p-value that will be interpreted.

If the p-value is LESS THAN .05, then there is significant agreement between the raters.

if the p-value is MORE THAN .05, then there is not agreement between the raters.

2. When interpreting Kappa, the following criteria are used:

0.40 - 0.59 is considered moderate agreement

0.60 - 0.79 is considered substantial agreement

0.80 - 0.99 is considered outstanding agreement


Click on the Validity button to continue.
Validity
Back to Inter-Rater Reliability
Research Engineer Home Page

Hire A Statistician - Statistical Consulting for Students

$75.00
Buy Now

DO YOU NEED TO HIRE A STATISTICIAN?

Eric Heidel, Ph.D. will provide the following statistical consulting services for undergraduate and graduate students at $75/hour. Secure checkout is available with Stripe, Venmo, Zelle, or PayPal.


  • Statistical Analysis 
  • Research Design
  • Sample Size Calculations
  • Diagnostic Testing and Epidemiological Calculations
  • Survey Design and Psychometrics

Contact Dr. Eric Heidel
consultation@scalelive.com
(865) 742-7731

Copyright © 2022 Scalë. All Rights Reserved. Patent Pending.