Correlations play a central role in applied psychometrics.

The inter-correlations among survey instrument items play a role in calculating internal consistency reliability coefficients (Cronbach's alpha, split-half, KR-20), test-retest reliability (Spearman-Brown formula), and inter-rater reliability (Kappa, ICC). Correlation matrices also play a significant role in principal components analysis (eigenvalues, factor loadings).

Correlations are used to generate convergent, predictive, and concurrent validity evidence. Significant correlations with theoretically or conceptually similar constructs/survey instruments denotes evidence of validity. In social sciences, a validity coefficient (or correlation coefficient) of .3 is considered evidence of validity.

Pearson's r and Spearman's rho are the most prevalent correlation tests used to generate validity evidence. These correlations are used with survey instruments that generate ordinal or continuous outcomes.

Logarithmic transformations are powerful statistical tools when employed and interpreted in the correct fashion. Transforming the distribution of a continuous variable due to violating normality allows researchers to account for outlying observations and use more powerful parametric statistics to assess any significant associations. 

Also, some continuous variables are naturally skewed.  One particular outcome that is prevalent in medicine is LOS or length of stay in the hospital.  Most patients will be in the hospital between one and three days, VERY FEW will be in the hospital for weeks and months at a time.  In order to include these outlying patients in analyses, transformations must be performed.  Naturally skewed variables can be analyzed with parametric statistics with transformations! 

An important thing to remember when conducting logarithmic transformations is that only the p-value associated with inferential statistics can be interpreted, NOT the means and standard deviations of the transformed observations. Instead, researchers should report the median and interquartile range for the distribution.

In instances where a phenomenon or outcome is less prevalent in the population, scientists are forced to work small sample sizes. It is just the nature of the science, and the phenomenon or outcome.

1. When working with smaller sample sizes, adequate statistical power (and therefore statistical significance) is VERY hard to achieve.

2. There is limited precision and accuracy when using categorical or ordinal outcomes, which can further decreases statistical power.

3. When measuring for small effect sizes, small sample sizes cannot provide enough variance in the outcome to detect clinically meaningful, but small effects. This REALLY decreases your statistical power since inferential statistics depend upon variance in the mathematical sense.

With this being said, remember to interpret the p-values yielded from RCT level studies with small sample sizes in the context of the aforementioned points. If a treatment effect does not obtain statistical significance, but appears to be CLINICALLY SIGNIFICANT with a p-value approaching significance (Type II error), then perhaps more credence can be found in the effect.

If researchers run bivariate tests on 30 different outcomes with less than 20 observations and claim significance without a Bonferroni adjustment, throw the article out.

There is a fundamental difference between chi-square and Fisher's Exact test. They are often used interchangeably both in everyday empirical discourse and also in the literature. There are many calculators available for free on the internet that will calculate inferential statistics for chi-square tests of independence and fisher's exact test. Without the proper statistical competencies, researchers can employ the wrong test. Here is how to know which of these tests to use with your research data:

1. Chi-square - This non-parametric test is used when you are looking at the association between dichotomous categorical variables. The primary inference yielded from this test is the unadjusted odds ratio with 95% confidence interval. EACH CELL of the 2x2 table MUST have at least five observations.

2. Fisher's Exact Test - This non-parametric test is employed when you are looking at the association between dichotomous categorical variables. The primary inference here is also the unadjusted odds ratio with 95% confidence interval. However, the Fisher's Exact Test is used instead of chi-square if ONE OF THE CELLS in the 2x2 has LESS than five observations.

Here is a really quick tip for making the statistics and outputs of SPSS much easier to interpret when using dichotomous predictor and outcome variables. Whatever "level" of the dichotomy that you are most interested in should be codified as a "1." If a participant has the characteristic or outcome of interest, codify those observations as "1" and the absence of the characteristic or outcome of interest as "0."  

SPSS has a default that always makes the highest numerical category be the reference group. However, most times, researchers want to know the odds of something occurring versus not occurring, NOT the odds of something not occurring versus the odds of it occurring. Therefore, it is important when running bivariate associations between dichotomous categorical variables to always use the codification scheme above so that the statistical outputs can be interpreted properly.

When conducting multivariate analyses, SPSS still uses the same reference default for the highest number category. The "point and click" interface for multivariate statistics in SPSS gives you the option to click on a "Categorical" button. Always do this and make sure that you set the category to "first" when running these types of statistics.  

As we continue our journey to break through the barriers associated with statistical lexicons, here is another dichotomy of popular statistical terms that are spoken commonly but not always understood by everyone.  

Parametric statistics are used to assess differences and effects for continuous outcomes. These statistical tests include one-sample t-tests, independent samples t-tests, one-way ANOVA, repeated-measures ANOVA, ANCOVA, factorial ANOVA, multiple regression, MANOVA, and MANCOVA. 

Non-parametric statistics are used to assess differences and effects for:

1. Ordinal outcomes - One-sample median tests, Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Friedman's ANOVA, Proportional odds regression

2. Categorical outcomes - Chi-square, Chi-square Goodness-of-fit, odds ratio, relative risk, McNemar's, Cochran's Q, Kaplan-Meier, log-rank test, Cochran-Mantel-Haenszel, Cox regression, logistic regression, multinomial logistic regression

3. Small sample sizes (n < 30) - Smaller sample sizes make it harder to meet the statistical assumptions associated with parametric statistics.  Non-parametric statistics can generate valid statistical inferences in these situations.

4. Violations of statistical assumptions for parametric tests - Normality, Homogeneity of variance, Normality of difference scores

When it comes to writing survey items that use Likert scales as response sets, use 5-point Likert scales with increasing order. The 5-point scale is preferable to a 4-point, 3-point, or dichotomous scales because there is more chance for variance with a 5-point scale and there is a "neutral" rating.

Variance in the responses is needed in order to properly assess the diversity that may exist in a population. Increased variance is also important for the underlying mathematics associated with reliability analysis, exploratory factor analysis, validity analysis, and confirmatory factor analysis.

The use of 5-point Likert scales also works well in an aesthetic fashion for structuring a survey. Space and time can be saved in survey administration when items from similar content areas use the same 5-point Likert response set. These questions can be formatted into a matrix.

Finally, increasing order should be used when using a Likert scale, going from left to right.  

For example:

Strongly Disagree, Disagree, Neither Agree Nor Disagree, Agree, Strongly Agree
Never, Rarely, Sometimes, Often, Always
Very Poor, Poor, Moderate, Good, Very Good

After my last post, I want to expand upon what the literature does to the person. One of my professors in graduate school said, "The literature changes you." At the time, I thought this was the dorkiest statement of all time. As a first year PhD student, I had NO IDEA what research constituted in regards to knowing the empirical literature.

The truth is, the literature DOES change you. It led me to fight an uphill battle for 6 years in the name of isomorphism. Come hell or high water, it will be published! (Manuscript currently under review with The Clinical Supervisor)

When I started my job as an assistant professor of biostatistics, I knew that I needed to get vested in the statistical consultation, evidence-based medicine, diagnostic testing, and epidemiology literature. One thing that really impacted me was an article that stated something to the effect of "The best biostatistical consultants are biostatistical scientists." 

Biostatistical scientists conduct collaborative as well as their own research. They provide high quality consultation to researchers from the inception to the publication of a research study. They teach courses related to empirical and statistical reasoning to residents, fellows, faculty, physicians, and staff. Lastly, and what really struck a chord with me, was that biostatistical scientists are supposed to invent new methods for applied practice.  

Understand something, I LOVE MATH.  And, I LOVE SCIENCE. But, mathematical notation and emerging mathematical theory are not within span of competencies. I came from a social science background where higher order mathematics are not requisite parts of the curricula. So, I knew that my conceptual and applied competencies related to math would not "cut it" in comparison to my fellow colleagues and academicians that specialize in these AWESOME FIELDS.  

Therefore, my idea/invention/new method would have to come from my conceptual and applied background. If you are reading this post, you are looking at the result of the literature's impact on me and a lot of hard work.  

I hope that it helps you in all of your future research endeavors.

EH

There is nothing more important when designing and conducting research than being heavily vested in the associated knowledge base. Research questions are born and formulated out of the literature. One cannot argue for a "gap" in the literature unless he or she has put forth the time and effort to know all of the literature. The literature also makes it very easy to make hard decisions in the preliminary phases of study planning.

Here is what the literature can do for you:

1. Give you an evidence-based measure of effect to use in an a priori power analysis. It will show more empirical rigor on your part if you use the values from the most current and highest-quality evidence available.

2. Help you choose the "gold standard" outcome that is most generalizable and applicable to your audience and peers. Using the best outcome measure available increases the internal validity of your study as well. If the same outcome is used in many studies, then it has more validity evidence to back it up. This, again, shows stronger empirical reasoning on your part.

3. Allow you to ask a question that is relevant and that will generate new knowledge. You will be able to pass the "So what?" question with ease when you know the literature. You will know what new knowledge needs to be generated and how it is relevant in the context of the existing literature.

4. Help you choose the correct research design to answer your research question. If you find that the literature only has observational evidence related to your area of interest, then you can make the informed decision to employ a more complex design to yield causal effects.   

Non-parametric tests like chi-square, fisher's exact test, Kruskal-Wallis, Cochran's Q, and Friedman's ANOVA do not have post hoc analyses to explain significant main effects. In order to conduct these post hoc anlayses, researchers have to resort to using subsequent non-parametric tests for two groups.

In a prior post, I explained how Mann-Whitney U tests were used in a post hoc fashion for significant main effects found with Kruskal-Wallis analyses. This is pertinent for between-subjects tests.

If you are using a within-subjects design with three or more observations of a dichotomous categorical outcome, you utilize Cochran's Q test to assess main effects. If a significant main effect is found, then McNemar's tests have to be employed for post hoc group comparisons. Significant post hoc tests (or relative risk calculations) will provide evidence of significant differences across observations or within-subjects.

Non-parametric statistics should be employed more often than they are in the literature. Many published studies use small sample sizes and ordinal or categorical outcomes. The statistical assumptions of more power parametric statistics can often not be met with these types of designs. Non-parametric statistics are robust to these violations and should be used accordingly. Post hoc analyses are important in non-parametric statistics, just like in parametric statistics.