Tags

  • Published on

    The assumption of independence of observations

    Generalized Estimating Equations (GEE) Homogeneity Of Variance Independence Of Observations Assumption Kurtosis Levene's Test Non-parametric Statistics Normality Outliers P-value Skewness Statistical Assumptions

    Independence of observations

    Each participant in a sample can only be counted as one observation

    As a biostatistician, I spend a lot of time testing for normality and homogeneity of variance.

    Skewness and kurtosis statistics are used to assess the normality of a continuous variable's distribution.  A skewness or kurtosis statistic above an absolute value of 2.0 is considered to be non-normal.  Distributions are often non-normal due to outliers in the distribution.  Any observation that falls more than 3.29 standard deviations away from the mean is considered an outlier.

    Levene's Test of Equality of Variances is used to measure for meeting the assumption of homogeneity of variance. Any Levene's Test with a p-value below .05 means that the assumption has been violated.  In the event that the assumption is violated, non-parametric tests can be employed.

    There is one more important statistical assumption that exists coincident with the aforementioned two, the assumption of independence of observations.  Simply stated, this assumption stipulates that study participants are independent of each other in the analysis. They are only counted once.

    In between-subjects designs, each study participant is a mutually exclusive observation that is completely independent from all other participants in all other groups.

    For within-subjects designs, each participant is independent of other participants.  There are just multiple observations of the outcome, per participant.

    With this being said, it is prevalent for researchers to take multiple measurements of an outcome and compare these multiple measurements in an independent fashion (oftentimes with differing numbers of observations across participants) or within-subjects (ALWAYS with differing numbers of observations of the outcome).  By default, these are not independent measures and violate the assumption of independence of observations.  What is one to do?

    The answer is generalized estimating equations (GEE).  This family of statistical tests are robust to multiple observations (or correlated observations) of an outcome and can be used for between-subjects, within-subjects, factorial, and multivariate analyses.
  • Published on

    Parametric vs. non-parametric statistics

    ANCOVA Test ANOVA Test Chi-square Chi-square Goodness-of-fit Cochran-Mantel-Haenszel Cochran's Q Test Cox Regression Factorial ANOVA Friedman's ANOVA Homogeneity Of Variance Independent Samples T-test Kruskal-Wallis Log-Rank Test Logistic Regression MANCOVA Mann-Whitney U MANOVA McNemar's Test Multinomial Logistic Regression Multiple Regression Non-parametric Statistics Normality Normality Of Difference Scores Odds Ratio With 95% CI One-sample Median Tests One-sample T-test Parametric Statistics Proportional Odds Regression Relative Risk Repeated-measures ANOVA Statistical Assumptions Wilcoxon

    Parametric statistics are more powerful statistics

    Non-parametric statistics are used with categorical and ordinal outcomes

    As we continue our journey to break through the barriers associated with statistical lexicons, here is another dichotomy of popular statistical terms that are spoken commonly but not always understood by everyone.  

    Parametric statistics are used to assess differences and effects for continuous outcomes. These statistical tests include one-sample t-tests, independent samples t-tests, one-way ANOVA, repeated-measures ANOVA, ANCOVA, factorial ANOVA, multiple regression, MANOVA, and MANCOVA. 

    Non-parametric statistics are used to assess differences and effects for:

    1. Ordinal outcomes - One-sample median tests, Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Friedman's ANOVA, Proportional odds regression

    2. Categorical outcomes - Chi-square, Chi-square Goodness-of-fit, odds ratio, relative risk, McNemar's, Cochran's Q, Kaplan-Meier, log-rank test, Cochran-Mantel-Haenszel, Cox regression, logistic regression, multinomial logistic regression

    3. Small sample sizes (n < 30) - Smaller sample sizes make it harder to meet the statistical assumptions associated with parametric statistics.  Non-parametric statistics can generate valid statistical inferences in these situations.

    4. Violations of statistical assumptions for parametric tests - Normality, Homogeneity of variance, Normality of difference scores
  • Published on

    McNemar's as a post hoc test for Cochran's Q

    Cochran's Q Test Kruskal-Wallis Mann-Whitney U McNemar's Test Non-parametric Statistics

    McNemar's can be used as a post hoc test

    Significant main effects for Cochran's Q need to be explained

    Non-parametric tests like chi-square, fisher's exact test, Kruskal-Wallis, Cochran's Q, and Friedman's ANOVA do not have post hoc analyses to explain significant main effects. In order to conduct these post hoc anlayses, researchers have to resort to using subsequent non-parametric tests for two groups.

    In a prior post, I explained how Mann-Whitney U tests were used in a post hoc fashion for significant main effects found with Kruskal-Wallis analyses. This is pertinent for between-subjects tests.

    If you are using a within-subjects design with three or more observations of a dichotomous categorical outcome, you utilize Cochran's Q test to assess main effects. If a significant main effect is found, then McNemar's tests have to be employed for post hoc group comparisons. Significant post hoc tests (or relative risk calculations) will provide evidence of significant differences across observations or within-subjects.

    Non-parametric statistics should be employed more often than they are in the literature. Many published studies use small sample sizes and ordinal or categorical outcomes. The statistical assumptions of more power parametric statistics can often not be met with these types of designs. Non-parametric statistics are robust to these violations and should be used accordingly. Post hoc analyses are important in non-parametric statistics, just like in parametric statistics. 
  • Published on

    Non-parametric statistics and small sample sizes

    Friedman's ANOVA Kruskal-Wallis Mann-Whitney U Non-parametric Statistics Psychometric Tests Sample Size Wilcoxon

    Non-parametric statistics are robust to small sample sizes

    The right way to conduct statistics

    Mark Twain said it best, "There are lies, damn lies, and statistics." Statistics can be misleading from both the standpoint of the person conducting the statistics and the person that is interpreting the analyses. Many between-subjects studies have small sample sizes (n < 20) and statistical assumptions for parametric statistics cannot be met.

    For basic researchers that operate day in and day out with small sample sizes, the answer is to use non-parametric statistics. Non-parametric statistical tests such as the Mann-Whitney U, Kruskal-Wallis, Wilcoxon, and Friedman's ANOVA are robust to violations of statistical assumptions and skewed distributions. These tests can yield interpretable medians, interquartile ranges, and p-values.

    Non-parametric statistics are also useful in the social sciences due to the inherent measurement error associated with assessing human behaviors, thoughts, feelings, intelligence, and emotional states. The underlying algebra associated with psychometrics relies on intercorrelations amongst constructs or items.  Correlations can easily be skewed by outlying observations and measurement error.  Therefore, in concordance with mathematical and empirical reasoning, non-parametric statistics should be used often for between-subjects comparisons of surveys, instruments, and psychological measures.
  • Published on

    Ordinal measures becoming continuous with normality

    ANOVA Test Chi-square Classical Test Theory Continuous Friedman's ANOVA Independent Samples T-test Item Response Theory Kruskal-Wallis Kurtosis Logistic Regression Mann-Whitney U Non-parametric Statistics Normality Ordinal Parametric Statistics Regression Repeated-measures ANOVA Skewness Wilcoxon

    Ordinal measures and normality

    Ordinal level measurement can become interval level with assumed normality

    Here is an interesting trick I picked up along the way when it comes to ordinal outcomes and some unvalidated measures. If you run skewness and kurtosis statistics on the ordinal variable and its distribution meets the assumption of normality (skewness and kurtosis statistics are less than an absolute value of 2.0), then you can "upgrade" the variable to a continuous level of measurement and analyze it using more powerful parametric statistics.  

    This type of thinking is the reason that the SAT, ACT, GRE, MCAT, LSAT, and validated psychological instruments are perceived at a continuous level. The scores yielded from these instruments, by definition, are not continuous because a "true zero" does not exist. Scores from these tests are often norm- or criterion-referenced to the population so that they can be interpreted in the correct context. Therefore, with the subjectivity and measurement error associated with classical test theory and item response theory, the scores are actually ordinal.

    With that being said, if the survey instrument or ordinal outcome is used in the empirical literature often and it meets the assumption of normality as per skewness and kurtosis statistics, treat the ordinal variable as a continuous variable and run analyses using parametric statistics (t-tests, ANOVA, regression) versus non-parametric statistics (Chi-square, Mann-Whitney U, Kruskal-Wallis, McNemar's, Wicoxon, Friedman's ANOVA, logistic regression). 
  • Published on

    Using naturally skewed continuous variables as outcome variables

    Kurtosis Listwise Deletion Logarithmic Transformations Non-parametric Statistics Outcome Outliers Skewness

    Transformed outcomes

    Some continuous variables will be naturally skewed

    In medicine, there is an important metric that signifies efficiency and quality in healthcare, length of stay (LOS) in the hospital. When thinking about the distribution of a variable such as LOS, you have to put it into a relative context. The vast majority of people will have an LOS of between 0-3 days given the type of treatment or injury that brought them to hospital. VERY FEW individuals will stay at the hospital one month, six months, or a year. Therefore, the distribution looks nothing like the normal curve and is extremely positively skewed.  

    As a researcher, you may want to predict for a continuous variable that has a natural and logical skewness to its distribution in the population. Yet, the assumption of normality is a central tenet of running statistical analyses. What is one to do in this situation?

    The answer is to first, run skewnessand kurtosis statistics to assess the normality of your continuous outcome.  If the either statistic is above an absolute value of 2.0, then the distribution is non-normal. Check for outliers in the distribution that are more than 3.29 standard deviations away from the mean. Make sure that the outlying observations were entered correctly.

    You now have a choice:

    1. You can delete the outlying observations in a listwise fashion. This should be done only if the number of outlying variables is less than 10% of the overall distribution. This is the least preferable choice.

    2. You can conduct a logarithmic transformation on the outcome variable. Doing this will normalize the distribution so that you can run the analysis using parametric statistics. The unstandardized beta coefficients, standard errors, and standardized beta coefficients are not interpretable, but the significance of the associations between the predictor variables and the transformed outcome can yield some inferential evidence.

    3. You can recode the continuous outcome variable into a lower level scale of measurement such as ordinal or categorical and run non-parametric statistics to seek out any associations. Of course, you are losing the precision and accuracy of continuous-level measurement and introducing measurement error into the outcome variable, but you will still be able to run inferential statistics.

    4. You can use non-parametric statistics without changing the skewed variable at all. That is one of the primary benefits of non-parametric statistics: They are robust to violations of normality and homogeneity of variance. Instead of interpreting means and standard deviations, you will interpret medians and interquartile ranges with non-parametric statistics. 

    Click on the Statistics button to learn more.