Tags

  • Published on

    Between-subjects statistics are used to compare independent groups

    Comparison of independent groups on an outcome

    Number of groups, scales of measurement, and meeting statistical assumptions

    Between-subjects statistics are used when comparing independent groups on an outcome. Independent groups means that the groups are "different" or "independent" from each other according to some characteristic. With between-subjects designs, participants can only be part of one group (independence) and only observed once (independence of observations, IOO).

    One chooses a between-subjects statistical test based on the following:

    1. Number of independent groups being compared (one group, two groups, or three or more groups)

    2. Scale of measurement of the outcome (categorical, ordinal, or continuous)

    3. Meeting statistical assumptions (independence of observations, normality, and homogeneity of variance)

    Here is a list of between-subjects statistical tests and when they are utilized in applied quantitative research:

    1. Chi-square Goodness-of-fit - One group, categorical outcome, a priori hypothesis for dispersal of outcome

    2. One-sample median test - One group, ordinal outcome, a priori hypothesis for median value

    3. One-sample t-test - One group, continuous outcome, meet the assumption of IOO and normality, a priori hypothesis for mean value

    4. Chi-square - Two independent groups, categorical outcome, and chi-square assumption (more than five observations in each cell)

    5. Fisher's Exact test - Two independent groups, categorical outcome, and when the chi-square assumption is not met

    6. Mann-Whitney U - Two independent groups, ordinal outcome, and when the assumption of homogeneity of variance for independent samples t-test is violated

    7. Independent samples t-test - Two independent groups, continuous outcome, meet the assumption of IOO, normality (skewness and kurtosis statistics), and homogeneity of variance (also known as homoscedasticity, tested with Levene's test)

    8. Unadjusted odds ratio - Three or more independent groups, categorical outcome, chi-square assumption, choose a reference category and compare each independent group to the reference

    9. Kruskal-Wallis - Three or more independent groups, ordinal outcome, and when the assumption of homogeneity of variance is violated

    10. ANOVA - Three or more independent groups, continuous outcome, meet the assumption of IOO, normality, and homogeneity of variance
  • Published on

    Intraclass Correlation Coefficient and inter-rater reliability

    Inter-rater reliability with continuous ratings

    Two or more raters giving multiple continuous ratings

    The Intraclass Correlation Coefficient (ICC) is a measure of inter-rater reliability that is used when two or more raters give ratings at a continuous level.  There are two factors that dictate what type of ICC model should be used in a given study.

    1.  Will the raters given ratings for all observations?

    2.  Are the raters a sample from the overall population or are the raters the only people in the population?

    When raters do not give ratings on all observations (i.e. three ratings are given from a random sampling of three raters out of a possible six independent raters), then the One-Way Random model is used.

    When raters give ratings for all observations (i.e. three ratings are given from three raters from the overall population for each observation), then the Two-Way Random model is used.

    When raters give ratings for all observations and the raters are the only valid members of the population (i.e. three ratings are given from the most esteemed scholars in an area), then the Two-Way Mixed model is used.