Tags

  • Published on

    Predictive validity is a powerful type of psychometric evidence

    Predictive Validity

    Correlations and regression are used to establish this kind of evidence

    Predictive validity evidence means that a survey instrument has the ability to predict some sort of occurrence in the future.  The most common application of predictive validity occurs in tests like the ACT, SAT, GRE, MCAT, LSAT, and GMAT. These tests are given before entering various phases of higher education to assess an individual's potential to graduate from either undergraduate or graduate school.  Interestingly enough, the correlation between these prevalent (and expensive) tests and graduation is only 0.3!  This means that 91% of what accounts for graduation is NOT associated with test scores on these instruments.  And we are talking a multi-BILLION dollar business...but, I digress.

    Predictive validity is calculated using simple correlation coefficients.  A correlation of 0.1 is considered weak evidence, a correlation of 0.3 denotes moderate evidence, and a correlation of 0.5 would make most social scientists jump for joy. Remember, in order to understand the amount of shared variance between two constructs, you simply "square" the correlation coefficient to yield the coefficient of determination.  Even with the highest level of predictive evidence with a predictive validity coefficient of 0.5, you are only accounting for 25% of the association between the two constructs!

    Within medicine, I believe that predictive validity plays an important role in imaging and early diagnosis.  One of the benefits of working in medicine is that the measures are more objective, concrete, observable, validated, and measurable versus the social sciences.  Correlations of 0.9 are common between various etiological, prognostic, confounding, clinical, and demographic phenomena within medicine.  If an imaging or diagnostic method can detect the earlier stages of a progressing disease state, then future outcomes can be mitigated with earlier and preventative treatment.  
  • Published on

    The role of correlations in psychometrics

    Correlations are used to generate validity evidence

    Concurrent, predictive, convergent, and divergent validity

    Correlations play a central role in applied psychometrics.

    The inter-correlations among survey instrument items play a role in calculating internal consistency reliability coefficients (Cronbach's alpha, split-half, KR-20), test-retest reliability (Spearman-Brown formula), and inter-rater reliability (Kappa, ICC). Correlation matrices also play a significant role in principal components analysis (eigenvalues, factor loadings).

    Correlations are used to generate convergent, predictive, and concurrent validity evidence. Significant correlations with theoretically or conceptually similar constructs/survey instruments denotes evidence of validity. In social sciences, a validity coefficient (or correlation coefficient) of .3 is considered evidence of validity.

    Pearson's r and Spearman's rho are the most prevalent correlation tests used to generate validity evidence. These correlations are used with survey instruments that generate ordinal or continuous outcomes.