Tags

  • Published on

    The role of correlations in psychometrics

    Concurrent Validity Convergent Validity Correlations Cronbach's Alpha ICC Inter-rater Reliability Internal Consistency Reliability Kappa Statistic KR-20 Pearson's R Predictive Validity Psychometric Tests Reliability Spearman-Brown Spearman's Rho Split-half Reliability Test-Retest Reliability Validity

    Correlations are used to generate validity evidence

    Concurrent, predictive, convergent, and divergent validity

    Correlations play a central role in applied psychometrics.

    The inter-correlations among survey instrument items play a role in calculating internal consistency reliability coefficients (Cronbach's alpha, split-half, KR-20), test-retest reliability (Spearman-Brown formula), and inter-rater reliability (Kappa, ICC). Correlation matrices also play a significant role in principal components analysis (eigenvalues, factor loadings).

    Correlations are used to generate convergent, predictive, and concurrent validity evidence. Significant correlations with theoretically or conceptually similar constructs/survey instruments denotes evidence of validity. In social sciences, a validity coefficient (or correlation coefficient) of .3 is considered evidence of validity.

    Pearson's r and Spearman's rho are the most prevalent correlation tests used to generate validity evidence. These correlations are used with survey instruments that generate ordinal or continuous outcomes.
  • Published on

    Logarithmic transformations for skewed variables

    Interquartile Range Logarithmic Transformations Median Normality P-value

    Logarithmic transformations adjust skewed distributions

    Analyze skewed data using more powerful parametric statistics

    Logarithmic transformations are powerful statistical tools when employed and interpreted in the correct fashion. Transforming the distribution of a continuous variable due to violating normality allows researchers to account for outlying observations and use more powerful parametric statistics to assess any significant associations. 

    Also, some continuous variables are naturally skewed.  One particular outcome that is prevalent in medicine is LOS or length of stay in the hospital.  Most patients will be in the hospital between one and three days, VERY FEW will be in the hospital for weeks and months at a time.  In order to include these outlying patients in analyses, transformations must be performed.  Naturally skewed variables can be analyzed with parametric statistics with transformations! 

    An important thing to remember when conducting logarithmic transformations is that only the p-value associated with inferential statistics can be interpreted, NOT the means and standard deviations of the transformed observations. Instead, researchers should report the median and interquartile range for the distribution.
  • Published on

    Small sample sizes, Type II errors, and empirical reasoning

    Accuracy Bonferroni Effect Size P-value Precision Sample Size Statistical-power-test Type II Error

    Small sample sizes can lead to Type II errors

    Significant effects may not be able to be detected

    In instances where a phenomenon or outcome is less prevalent in the population, scientists are forced to work small sample sizes. It is just the nature of the science, and the phenomenon or outcome.

    1. When working with smaller sample sizes, adequate statistical power (and therefore statistical significance) is VERY hard to achieve.

    2. There is limited precision and accuracy when using categorical or ordinal outcomes, which can further decreases statistical power.

    3. When measuring for small effect sizes, small sample sizes cannot provide enough variance in the outcome to detect clinically meaningful, but small effects. This REALLY decreases your statistical power since inferential statistics depend upon variance in the mathematical sense.

    With this being said, remember to interpret the p-values yielded from RCT level studies with small sample sizes in the context of the aforementioned points. If a treatment effect does not obtain statistical significance, but appears to be CLINICALLY SIGNIFICANT with a p-value approaching significance (Type II error), then perhaps more credence can be found in the effect.

    If researchers run bivariate tests on 30 different outcomes with less than 20 observations and claim significance without a Bonferroni adjustment, throw the article out.
  • Published on

    Chi-square vs. Fisher's Exact Test

    Chi-square Fisher's Exact Tests

    Chi-square vs. Fisher's Exact Test

    Meeting chi-square assumption of at least five observations per cell

    There is a fundamental difference between chi-square and Fisher's Exact test. They are often used interchangeably both in everyday empirical discourse and also in the literature. There are many calculators available for free on the internet that will calculate inferential statistics for chi-square tests of independence and fisher's exact test. Without the proper statistical competencies, researchers can employ the wrong test. Here is how to know which of these tests to use with your research data:

    1. Chi-square - This non-parametric test is used when you are looking at the association between dichotomous categorical variables. The primary inference yielded from this test is the unadjusted odds ratio with 95% confidence interval. EACH CELL of the 2x2 table MUST have at least five observations.

    2. Fisher's Exact Test - This non-parametric test is employed when you are looking at the association between dichotomous categorical variables. The primary inference here is also the unadjusted odds ratio with 95% confidence interval. However, the Fisher's Exact Test is used instead of chi-square if ONE OF THE CELLS in the 2x2 has LESS than five observations.
  • Published on

    Dichotomous variables in SPSS

    Categorical Variables

    Analyze dichotomous variables in SPSS

    Choose reference categories or dummy code variables

    Here is a really quick tip for making the statistics and outputs of SPSS much easier to interpret when using dichotomous predictor and outcome variables. Whatever "level" of the dichotomy that you are most interested in should be codified as a "1." If a participant has the characteristic or outcome of interest, codify those observations as "1" and the absence of the characteristic or outcome of interest as "0."  

    SPSS has a default that always makes the highest numerical category be the reference group. However, most times, researchers want to know the odds of something occurring versus not occurring, NOT the odds of something not occurring versus the odds of it occurring. Therefore, it is important when running bivariate associations between dichotomous categorical variables to always use the codification scheme above so that the statistical outputs can be interpreted properly.

    When conducting multivariate analyses, SPSS still uses the same reference default for the highest number category. The "point and click" interface for multivariate statistics in SPSS gives you the option to click on a "Categorical" button. Always do this and make sure that you set the category to "first" when running these types of statistics.  
  • Published on

    Parametric vs. non-parametric statistics

    ANCOVA Test ANOVA Test Chi-square Chi-square Goodness-of-fit Cochran-Mantel-Haenszel Cochran's Q Test Cox Regression Factorial ANOVA Friedman's ANOVA Homogeneity Of Variance Independent Samples T-test Kruskal-Wallis Log-Rank Test Logistic Regression MANCOVA Mann-Whitney U MANOVA McNemar's Test Multinomial Logistic Regression Multiple Regression Non-parametric Statistics Normality Normality Of Difference Scores Odds Ratio With 95% CI One-sample Median Tests One-sample T-test Parametric Statistics Proportional Odds Regression Relative Risk Repeated-measures ANOVA Statistical Assumptions Wilcoxon

    Parametric statistics are more powerful statistics

    Non-parametric statistics are used with categorical and ordinal outcomes

    As we continue our journey to break through the barriers associated with statistical lexicons, here is another dichotomy of popular statistical terms that are spoken commonly but not always understood by everyone.  

    Parametric statistics are used to assess differences and effects for continuous outcomes. These statistical tests include one-sample t-tests, independent samples t-tests, one-way ANOVA, repeated-measures ANOVA, ANCOVA, factorial ANOVA, multiple regression, MANOVA, and MANCOVA. 

    Non-parametric statistics are used to assess differences and effects for:

    1. Ordinal outcomes - One-sample median tests, Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Friedman's ANOVA, Proportional odds regression

    2. Categorical outcomes - Chi-square, Chi-square Goodness-of-fit, odds ratio, relative risk, McNemar's, Cochran's Q, Kaplan-Meier, log-rank test, Cochran-Mantel-Haenszel, Cox regression, logistic regression, multinomial logistic regression

    3. Small sample sizes (n < 30) - Smaller sample sizes make it harder to meet the statistical assumptions associated with parametric statistics.  Non-parametric statistics can generate valid statistical inferences in these situations.

    4. Violations of statistical assumptions for parametric tests - Normality, Homogeneity of variance, Normality of difference scores