Statistical Consultation Line: (865) 742-7731
Accredited Professional Statistician For Hire
  • Contact Form

The role of correlations in psychometrics

11/29/2014

0 Comments

 

Correlations are used to generate validity evidence

Concurrent, predictive, convergent, and divergent validity

Correlations play a central role in applied psychometrics.

The inter-correlations among survey instrument items play a role in calculating internal consistency reliability coefficients (Cronbach's alpha, split-half, KR-20), test-retest reliability (Spearman-Brown formula), and inter-rater reliability (Kappa, ICC). Correlation matrices also play a significant role in principal components analysis (eigenvalues, factor loadings).

Correlations are used to generate convergent, predictive, and concurrent validity evidence. Significant correlations with theoretically or conceptually similar constructs/survey instruments denotes evidence of validity. In social sciences, a validity coefficient (or correlation coefficient) of .3 is considered evidence of validity.

Pearson's r and Spearman's rho are the most prevalent correlation tests used to generate validity evidence. These correlations are used with survey instruments that generate ordinal or continuous outcomes.

Scale, LLC
0 Comments

Logarithmic transformations for skewed variables

11/23/2014

0 Comments

 

Logarithmic transformations adjust skewed distributions

Analyze skewed data using more powerful parametric statistics

Logarithmic transformations are powerful statistical tools when employed and interpreted in the correct fashion. Transforming the distribution of a continuous variable due to violating normality allows researchers to account for outlying observations and use more powerful parametric statistics to assess any significant associations. 

Also, some continuous variables are naturally skewed.  One particular outcome that is prevalent in medicine is LOS or length of stay in the hospital.  Most patients will be in the hospital between one and three days, VERY FEW will be in the hospital for weeks and months at a time.  In order to include these outlying patients in analyses, transformations must be performed.  Naturally skewed variables can be analyzed with parametric statistics with transformations! 

An important thing to remember when conducting logarithmic transformations is that only the p-value associated with inferential statistics can be interpreted, NOT the means and standard deviations of the transformed observations. Instead, researchers should report the median and interquartile range for the distribution.

Scale, LLC
0 Comments

Small sample sizes, Type II errors, and empirical reasoning

11/18/2014

0 Comments

 

Small sample sizes can lead to Type II errors

Significant effects may not be able to be detected

In instances where a phenomenon or outcome is less prevalent in the population, scientists are forced to work small sample sizes. It is just the nature of the science, and the phenomenon or outcome.

1. When working with smaller sample sizes, adequate statistical power (and therefore statistical significance) is VERY hard to achieve.

2. There is limited precision and accuracy when using categorical or ordinal outcomes, which can further decreases statistical power.

3. When measuring for small effect sizes, small sample sizes cannot provide enough variance in the outcome to detect clinically meaningful, but small effects. This REALLY decreases your statistical power since inferential statistics depend upon variance in the mathematical sense.

With this being said, remember to interpret the p-values yielded from RCT level studies with small sample sizes in the context of the aforementioned points. If a treatment effect does not obtain statistical significance, but appears to be CLINICALLY SIGNIFICANT with a p-value approaching significance (Type II error), then perhaps more credence can be found in the effect.

If researchers run bivariate tests on 30 different outcomes with less than 20 observations and claim significance without a Bonferroni adjustment, throw the article out.

Scale, LLC
0 Comments

Chi-square vs. Fisher's Exact Test

11/17/2014

0 Comments

 

Chi-square vs. Fisher's Exact Test

Meeting chi-square assumption of at least five observations per cell

There is a fundamental difference between chi-square and Fisher's Exact test. They are often used interchangeably both in everyday empirical discourse and also in the literature. There are many calculators available for free on the internet that will calculate inferential statistics for chi-square tests of independence and fisher's exact test. Without the proper statistical competencies, researchers can employ the wrong test. Here is how to know which of these tests to use with your research data:

1. Chi-square - This non-parametric test is used when you are looking at the association between dichotomous categorical variables. The primary inference yielded from this test is the unadjusted odds ratio with 95% confidence interval. EACH CELL of the 2x2 table MUST have at least five observations.

2. Fisher's Exact Test - This non-parametric test is employed when you are looking at the association between dichotomous categorical variables. The primary inference here is also the unadjusted odds ratio with 95% confidence interval. However, the Fisher's Exact Test is used instead of chi-square if ONE OF THE CELLS in the 2x2 has LESS than five observations.

Scale, LLC
0 Comments

Dichotomous variables in SPSS

11/16/2014

0 Comments

 

Analyze dichotomous variables in SPSS

Choose reference categories or dummy code variables

Here is a really quick tip for making the statistics and outputs of SPSS much easier to interpret when using dichotomous predictor and outcome variables. Whatever "level" of the dichotomy that you are most interested in should be codified as a "1." If a participant has the characteristic or outcome of interest, codify those observations as "1" and the absence of the characteristic or outcome of interest as "0."  

SPSS has a default that always makes the highest numerical category be the reference group. However, most times, researchers want to know the odds of something occurring versus not occurring, NOT the odds of something not occurring versus the odds of it occurring. Therefore, it is important when running bivariate associations between dichotomous categorical variables to always use the codification scheme above so that the statistical outputs can be interpreted properly.

When conducting multivariate analyses, SPSS still uses the same reference default for the highest number category. The "point and click" interface for multivariate statistics in SPSS gives you the option to click on a "Categorical" button. Always do this and make sure that you set the category to "first" when running these types of statistics.  

Scale, LLC
0 Comments

Parametric vs. non-parametric statistics

11/14/2014

0 Comments

 

Parametric statistics are more powerful statistics

Non-parametric statistics are used with categorical and ordinal outcomes

As we continue our journey to break through the barriers associated with statistical lexicons, here is another dichotomy of popular statistical terms that are spoken commonly but not always understood by everyone.  

Parametric statistics are used to assess differences and effects for continuous outcomes. These statistical tests include one-sample t-tests, independent samples t-tests, one-way ANOVA, repeated-measures ANOVA, ANCOVA, factorial ANOVA, multiple regression, MANOVA, and MANCOVA. 

Non-parametric statistics are used to assess differences and effects for:

1. Ordinal outcomes - One-sample median tests, Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Friedman's ANOVA, Proportional odds regression

2. Categorical outcomes - Chi-square, Chi-square Goodness-of-fit, odds ratio, relative risk, McNemar's, Cochran's Q, Kaplan-Meier, log-rank test, Cochran-Mantel-Haenszel, Cox regression, logistic regression, multinomial logistic regression

3. Small sample sizes (n < 30) - Smaller sample sizes make it harder to meet the statistical assumptions associated with parametric statistics.  Non-parametric statistics can generate valid statistical inferences in these situations.

4. Violations of statistical assumptions for parametric tests - Normality, Homogeneity of variance, Normality of difference scores

Scale, LLC
0 Comments

Writing survey items

11/10/2014

0 Comments

 

Write survey items that cover content areas

Survey items are composed of item stems and response sets

When it comes to writing survey items that use Likert scales as response sets, use 5-point Likert scales with increasing order. The 5-point scale is preferable to a 4-point, 3-point, or dichotomous scales because there is more chance for variance with a 5-point scale and there is a "neutral" rating.

Variance in the responses is needed in order to properly assess the diversity that may exist in a population. Increased variance is also important for the underlying mathematics associated with reliability analysis, exploratory factor analysis, validity analysis, and confirmatory factor analysis.

The use of 5-point Likert scales also works well in an aesthetic fashion for structuring a survey. Space and time can be saved in survey administration when items from similar content areas use the same 5-point Likert response set. These questions can be formatted into a matrix.

Finally, increasing order should be used when using a Likert scale, going from left to right.  

For example:

Strongly Disagree, Disagree, Neither Agree Nor Disagree, Agree, Strongly Agree
Never, Rarely, Sometimes, Often, Always
Very Poor, Poor, Moderate, Good, Very Good

Scale, LLC
0 Comments

Biostatistical scientists

11/9/2014

0 Comments

 

Biostatistical scientists bolster the validity of research

More empirical rigor, precision and accuracy, and internal and external validity

After my last post, I want to expand upon what the literature does to the person. One of my professors in graduate school said, "The literature changes you." At the time, I thought this was the dorkiest statement of all time. As a first year PhD student, I had NO IDEA what research constituted in regards to knowing the empirical literature.

The truth is, the literature DOES change you. It led me to fight an uphill battle for 6 years in the name of isomorphism. Come hell or high water, it will be published! (Manuscript currently under review with The Clinical Supervisor)

When I started my job as an assistant professor of biostatistics, I knew that I needed to get vested in the statistical consultation, evidence-based medicine, diagnostic testing, and epidemiology literature. One thing that really impacted me was an article that stated something to the effect of "The best biostatistical consultants are biostatistical scientists." 

Biostatistical scientists conduct collaborative as well as their own research. They provide high quality consultation to researchers from the inception to the publication of a research study. They teach courses related to empirical and statistical reasoning to residents, fellows, faculty, physicians, and staff. Lastly, and what really struck a chord with me, was that biostatistical scientists are supposed to invent new methods for applied practice.  

Understand something, I LOVE MATH.  And, I LOVE SCIENCE. But, mathematical notation and emerging mathematical theory are not within span of competencies. I came from a social science background where higher order mathematics are not requisite parts of the curricula. So, I knew that my conceptual and applied competencies related to math would not "cut it" in comparison to my fellow colleagues and academicians that specialize in these AWESOME FIELDS.  

Therefore, my idea/invention/new method would have to come from my conceptual and applied background. If you are reading this post, you are looking at the result of the literature's impact on me and a lot of hard work.  

I hope that it helps you in all of your future research endeavors.

EH

Scale, LLC
0 Comments

Mastery of the literature

11/6/2014

0 Comments

 

Mastery of the literature leads to relevant research questions

Become an expert in the empirical field of endeavor

There is nothing more important when designing and conducting research than being heavily vested in the associated knowledge base. Research questions are born and formulated out of the literature. One cannot argue for a "gap" in the literature unless he or she has put forth the time and effort to know all of the literature. The literature also makes it very easy to make hard decisions in the preliminary phases of study planning.

Here is what the literature can do for you:

1. Give you an evidence-based measure of effect to use in an a priori power analysis. It will show more empirical rigor on your part if you use the values from the most current and highest-quality evidence available.

2. Help you choose the "gold standard" outcome that is most generalizable and applicable to your audience and peers. Using the best outcome measure available increases the internal validity of your study as well. If the same outcome is used in many studies, then it has more validity evidence to back it up. This, again, shows stronger empirical reasoning on your part.

3. Allow you to ask a question that is relevant and that will generate new knowledge. You will be able to pass the "So what?" question with ease when you know the literature. You will know what new knowledge needs to be generated and how it is relevant in the context of the existing literature.

4. Help you choose the correct research design to answer your research question. If you find that the literature only has observational evidence related to your area of interest, then you can make the informed decision to employ a more complex design to yield causal effects.   

Scale, LLC
0 Comments

McNemar's as a post hoc test for Cochran's Q

11/5/2014

0 Comments

 

McNemar's can be used as a post hoc test

Significant main effects for Cochran's Q need to be explained

Non-parametric tests like chi-square, fisher's exact test, Kruskal-Wallis, Cochran's Q, and Friedman's ANOVA do not have post hoc analyses to explain significant main effects. In order to conduct these post hoc anlayses, researchers have to resort to using subsequent non-parametric tests for two groups.

In a prior post, I explained how Mann-Whitney U tests were used in a post hoc fashion for significant main effects found with Kruskal-Wallis analyses. This is pertinent for between-subjects tests.

If you are using a within-subjects design with three or more observations of a dichotomous categorical outcome, you utilize Cochran's Q test to assess main effects. If a significant main effect is found, then McNemar's tests have to be employed for post hoc group comparisons. Significant post hoc tests (or relative risk calculations) will provide evidence of significant differences across observations or within-subjects.

Non-parametric statistics should be employed more often than they are in the literature. Many published studies use small sample sizes and ordinal or categorical outcomes. The statistical assumptions of more power parametric statistics can often not be met with these types of designs. Non-parametric statistics are robust to these violations and should be used accordingly. Post hoc analyses are important in non-parametric statistics, just like in parametric statistics. 

Scale, LLC
0 Comments
<<Previous

    Archives

    March 2016
    January 2016
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014

    Author

    Eric Heidel, Ph.D. is Owner and Operator of Scalë, LLC.

    Categories

    All
    95% Confidence Interval
    Absolute Risk Reduction
    Accuracy
    Acquiring Clinical Evidence
    Adjusted Odds Ratio
    Affordable Care Act
    Alpha Value
    ANCOVA Test
    ANOVA Test
    Applying Clinical Evidence
    Appraisal Of The Literature
    Appraising Clinical Evidence
    A Priori
    Area Under The Curve
    Asking Clinical Questions
    Assessing Clinical Practice
    AUC
    Basic Science
    Beta Value
    Between-subjects
    Biserial
    Blinding
    Bloom's Taxonomy
    Bonferroni
    Boolean Operators
    Calculator
    Case-control Design
    Case Series
    Categorical
    Causal Effects
    Chi-square
    Chi-square Assumption
    Chi-square Goodness-of-fit
    Classical Test Theory
    Clinical Pathways
    Clustered Random Sampling
    Cochran-Mantel-Haenszel
    Cochran's Q Test
    Coefficient Of Determination
    Cognitive Dissonance
    Cohort
    Comparative Effectiveness Research
    Comparator
    Concurrent Validity
    Confidence Interval
    Confirmatory Factor Analysis
    Construct Specification
    Construct Validity
    Continuous
    Control Event Rate
    Convenience Sampling Method
    Convergent Validity
    Copyright
    Correlations
    Count Variables
    Cox Regression
    Cronbach's Alpha
    Cross-sectional
    Curriculum Vitae
    Database Management
    Diagnostic Testing
    EBM
    Education
    Effect Size
    Empirical Literature
    Epidemiology
    Equivalency Trial
    Eric Heidel
    Evidence-based Medicine
    Exclusion Criteria
    Experimental Designs
    Experimental Event Rate
    Facebook
    Factorial ANOVA
    Feasible Research Questions
    FINER
    Fisher's Exact Tests
    Friedman's ANOVA
    Generalized Estimating Equations (GEE)
    "gold Standard" Outcome
    G*Power
    Guidelines For Authors
    Hazard Ratio
    Hierarchical Regression
    Homogeneity Of Variance
    Hypothesis Testing
    ICC
    Incidence
    Inclusion Criteria
    Independence Of Observations Assumption
    Independent Samples T-test
    Intention-to-treat
    Internal Consistency Reliability
    Interquartile Range
    Inter-rater Reliability
    Interval Variables
    Intervention
    Intraclass Correlation Coefficient
    Isomorphism
    Item Response Theory
    Kaplan-Meier Curve
    Kappa Statistic
    KR-20
    Kruskal-Wallis
    Kurtosis
    Levene's Test
    Likert Scales
    Linearity
    Listwise Deletion
    Logarithmic Transformations
    Logistic Regression
    Log-Rank Test
    Longitudinal Data
    MANCOVA
    Mann-Whitney U
    MANOVA
    Mass Emails In Survey Research
    Math
    Mauchly's Test
    McNemar's Test
    Mean
    Measurement
    Median
    Medicine
    Merging Databases
    Missing Data
    Mode
    Multinomial Logistic Regression
    Multiple Regression
    Multivariate Statistics
    Negative Binomial Regression
    Negative Predictive Value
    Nominal Variables
    Nonequivalent Control Group Design
    Non-inferiority
    Non-inferiority Trial
    Non-parametric Statistics
    Non-probability Sampling
    Normality
    Normality Of Difference Scores
    Normal Probability Plot
    Novel Research Question
    Number Needed To Treat
    Observational Research
    Odds Ratio With 95% CI
    One-sample Median Tests
    One-sample T-test
    One-sided Hypothesis
    One-Way Random
    Operationalization
    Ordinal
    Outcome
    Outliers
    Parametric Statistics
    Pearson's R
    Ph.D.
    Phi Coefficient
    PICO
    Pilot Study
    Point Biserial
    Poisson Regression
    Population
    Positive Predictive Value
    Post Hoc
    Post-positivism
    PPACA
    PPV
    Precision
    Predictive Validity
    Prevalence
    Principal Components Analysis
    Probability Sampling
    Propensity Score Matching
    Proportion
    Proportional Odds Regression
    Prospective Cohort
    Psychometrics
    Psychometric Tests
    Publication
    Publication Bias
    Purposive Sampling
    P-value
    Random Assignment
    Randomized Controlled Trial
    Random Selection
    Rank Biserial
    Ratio Variables
    Receiver Operator Characteristic
    Regression
    Regression Analysis
    Relative Risk
    Relevant Research Question
    Reliability
    Repeated-measures ANOVA
    Repeated-measures T-test
    Research
    Research Design
    Research Engineer
    Research Journal
    Research Question
    Residual Analysis
    Retrospective Cohort
    ROC Curve
    Sample Size
    Sampling
    Sampling Error
    Sampling Method
    Scales Of Measurement
    Science
    Search Engine
    Search Query
    Sensitivity
    Simple Random Sampling
    Sitemap
    Skewness
    Social Science
    Spearman-Brown
    Spearman's Rho
    Specificity
    Specificity In Literature Searching
    Sphericity Assumption
    Split-half Reliability
    SPSS
    Standard Deviation
    Standards Of Care
    Statistical Analysis
    Statistical Assumptions
    Statistical Consultation
    Statistical Power
    Statistical Power Analysis
    Statistical-power-test
    Statistician
    Statistics
    Stratified Random Sampling
    Survey
    Survey Construct Specification
    Survey Methods
    Systematic Review
    Test-Retest Reliability
    Twitter
    Two-sided Hypothesis
    Two-Way Mixed
    Two-Way Random
    Type I Error
    Type II Error
    Unadjusted Odds Ratio
    Validity
    Variables
    Variance
    Wilcoxon
    Within-subjects
    YouTube


    Contact Form

Contact Dr. Eric Heidel
consultation@scalelive.com
(865) 742-7731

Copyright © 2023 Scalë. All Rights Reserved. Patent Pending.