Tags

  • Published on

    Operationalization of constructs and behaviors

    Affordable Care Act Confirmatory Factor Analysis Construct Validity Cross-sectional Operationalization Prevalence Principal Components Analysis Social Science Standards Of Care Survey

    Operationalization leading to understanding

    Measurement of new phenomena

    The term operationalization is very near and dear to my heart since I conducted my dissertation on operationalizing and validating the construct of isomorphism in supervision. Operationalization essentially means defining observable and measurable components of a given construct or behavior.

    The term is used often in the social sciences because scientists in that field have to spend so much time creating and validating their constructs of interest, just to be able to measure for them. From an empirical standpoint, they have to operationalize the construct as it exists within the perception, context, experience, and environment of members of a population.

    Many social scientists use survey methodologies (cross-sectional) to operationalize an abstract, new, or unique construct or behavior. They master the content area related to the construct, create a survey, and then administer it to a sample from a targeted population to see what content areas or items account for the most variance. Principal components analysis and confirmatory factor analysis are used to establish the construct validity of survey instruments.

    Medical professionals use cross-sectional research designs to establish the prevalence of disease states. Operationalization within physiology deals more with using "gold standard" techniques and concrete measures such as lab values.  Treatment protocols are another form of operationalization within medicine.  Certain procedures like a central line insertion require 20+ sequential steps to be conducted by surgical team members, every time.  With the advent of the Affordable Care Act and upcoming clinical pathways, operationalization will play an even larger role in building economical, efficient, and effective standards of care.    
  • Published on

    Evidence-based medicine and its applications

    Acquiring Clinical Evidence Applying Clinical Evidence Appraisal Of The Literature Appraising Clinical Evidence Asking Clinical Questions Assessing Clinical Practice Bloom's Taxonomy Cognitive Dissonance EBM Evidence-based Medicine PICO

    Critical appraisal of the clinical evidence

    The cart before the horse

    I'm getting ready to add an Education section to the website, I decided to go back to first principles.  Bloom's Taxonomy had a pervasive impact on my philosophy of learning, teaching, and cognitive complexity.  I used it back in February of this year for an evidence-based medicine (EBM) presentation at work.  Bloom's Taxonomy* stipulated six levels of "knowing" or cognitive complexity.  The six levels in increasing order were knowledge, comprehension, application, analysis, synthesis, and evaluation.  

    Here is the conundrum that Bloom's Taxonomy exacts upon applied EBM practice:

    There are five steps to EBM:  Asking, acquiring, appraising, applying, and assessing.  

    With asking, the EBM literature posits that clinicians experience "cognitive dissonance" when they have a knowledge gap in their clinical practice.  In order to deter the dissonance, the clinician decides to ask a clinical question to fill that gap.

    With acquiring, the clinician uses the PICO (population, intervention, comparator, outcome) mnemonic to acquire the best clinical evidence, given the resources and time available.

    Now we get to critical appraisal of the literature.  When looking at the nomenclature of the word "appraisal," it is reflective of the highest level of "knowing" or cognitive complexity in Bloom's Taxonomy, evaluation.  EBM stipulates that clinicians must be able to critically appraise the methods and statistical analyses of published studies.  This means that clinicians have to have functioning at a very high cognitive level to do this correctly.

    However, past literature has shown that researchers feel anxious and intimidated by statistics due to a lack of experience and competency.**  Also, undergraduate and graduate medical training rarely equips clinicians with the necessary competencies to conduct and effectively interpret clinical research evidence.***

    So, how can your everyday clinician with limited empirical/statistical training who feels "cognitive dissonance" a second time in the five steps of EBM critically appraise the literature?  Therein lies the conundrum, in my opinion.  

    I'm positing that we need to refocus our efforts on the lower echelons of Bloom's Taxonomy by educating physicians, residents, fellows, faculty, pharmacists, nurses, and staff to better understand (knowledge), recognize (comprehension), choose (application), examine (analysis), and design (synthesis) research studies before we can expect them to critically appraise (evaluation) the literature.  
    *Bloom, B. S.; Engelhart, M. D.; Furst, E. J.; Hill, W. H.;Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.
    **Marquardt, DW.  Criteria for evaluating the performance of statistical consultants in industry.  The American Statistician 1981; 35; 216-219.
    ***Wegwarth O.  Statistical illiteracy in residents:  What they do not learn today will hurt their patients tomorrow.  Journal of Graduate Medical Education 2013; 5; 340-341.
  • Published on

    Preliminary statistical consultation

    Database Management Research Engineer Sample Size Statistical Analysis Statistical Consultation Statistician Statistics

    Support your local statistician!

    Seek out methodological and statistical consultation

    If you have access to a statistical consultants or statisticians within your empirical or clinical environment, seek out their services in the preliminary phases of planning your study. Here is a list of things that I do for residents, fellows, faculty, physicians, pharmacists, nurses, and staff at an academic regional medical campus:

    1. Sample Size - I conduct sample size calculations for at least of 80-85% of my first-time clients. They often want to know how many people they need to reach a significant p-value. We work through the process of acquiring an evidence-based measure of effect that reflects what their research question is trying to answer.

    It feels good knowing that you have a good chance of detecting significance with a small sample size. Also, it is good to find out that you have to collect A LOT more observations than you thought you would. Post hoc power analyses should be run for any non-significant main effects that may be considered Type II errors (limited or small sample sizes).

    2. Statistical analysis - Real biostatistical scientists and statisticians will conduct your statistical analyses in an objective and expeditious manner to help you answer your research questions. Please help them understand what your research question is and what research design you want to use to answer it to the best of your abilities. They will be able to help you choose the correct statistic given that you can tell them the scale of measurement for your primary outcome and what type of design (between-subjects, within-subjects, correlational, mixed, or multivariate) you want to use to answer your question. It is also important to know WHO or WHAT you want to include in your sample in terms of inclusion and exclusion criteria. Finally, know your content area. We may not know your knowledge/philosophical base and need to understand the entire picture, as much as you can tell us.

    3. Database management - Go ahead and let us build your database in a basic Excel spreadsheet and send an accompanying code book in Word so that we are all on the same page. It helps us all know what is going on, what variables are being collected, what they mean, how they are measured, and how the analysis will work. Share it with all members of the research team. Use the code book when entering your data. Tell the rest of us if you make changes to the code book or database. These simple tasks and communicative efforts can mean the difference between your statistics being run in five minutes versus five weeks.  SERIOUSLY.

    4. Write-up of findings for publication - We will give you an annotated write-up of your findings with statistical outputs and give you basic and unbiased interpretations of the statistical results of your study. We can help you write up the statistical methods and results sections of your abstracts and manuscripts. We can even help you design tables and graphs that will make your study findings more aesthetically and visually appealing to your audience.

    When it comes to authorship, if you feel that your statistical professional's contribution to the design, execution, and interpretation of your study warrants authorship, offer it to them. They will greatly appreciate it! However, YOU SHOULD NEVER BE REQUIRED TO GIVE US AUTHORSHIP JUST BECAUSE WE RAN YOUR STATISTICS FOR YOU.  IT IS UNETHICAL FOR US TO REQUIRE AUTHORSHIP FOR DOING OUR JOB. THAT IS, IF OUR JOB IS TO RUN STATISTICS IN YOUR EMPIRICAL OR CLINICAL ENVIRONMENT.          
  • Published on

    G*Power for the masses

    Effect Size G*Power Sample Size Sampling Statistical Power Analysis

    G*Power is a necessary tool for every researcher's toolkit

    Easy statistical power and sample size calculations

    I'm trying to run an online business so I'm fully Google-integrated. I see that there many search queries of different derivations related to sample size calculation as it relates to behind-the-scenes tracking measures.

    There is an open-source tool available to EVERYONE that allows you to calculate your own a priori and post hoc power analyses. It is called G*Power and as your personal statistical consultant, I highly suggest you go to the following web address and download Version 3.0 to your respective device:

    http://www.gpower.hhu.de/en.html    

    The researchers that developed this program have made a great contribution to science. It is truly a great and FREE program that can run a litany of different power analyses. You can find out in minutes how large of a sample size that you need, given that you have an idea of the effect size that you are attempting to detect in your study.

    Use means, proportions, and variance measures from published studies in your field to have the most empirically rigorous hypothesized effect. Enter these values into G*Power and the adjust the variance and magnitude of the effect size to see how the required sample size changes.   

    Click on the Sample Size button to access the methods of conducting and interpreting sample size calculations for ten different statistical tests.
  • Published on

    Using naturally skewed continuous variables as outcome variables

    Kurtosis Listwise Deletion Logarithmic Transformations Non-parametric Statistics Outcome Outliers Skewness

    Transformed outcomes

    Some continuous variables will be naturally skewed

    In medicine, there is an important metric that signifies efficiency and quality in healthcare, length of stay (LOS) in the hospital. When thinking about the distribution of a variable such as LOS, you have to put it into a relative context. The vast majority of people will have an LOS of between 0-3 days given the type of treatment or injury that brought them to hospital. VERY FEW individuals will stay at the hospital one month, six months, or a year. Therefore, the distribution looks nothing like the normal curve and is extremely positively skewed.  

    As a researcher, you may want to predict for a continuous variable that has a natural and logical skewness to its distribution in the population. Yet, the assumption of normality is a central tenet of running statistical analyses. What is one to do in this situation?

    The answer is to first, run skewnessand kurtosis statistics to assess the normality of your continuous outcome.  If the either statistic is above an absolute value of 2.0, then the distribution is non-normal. Check for outliers in the distribution that are more than 3.29 standard deviations away from the mean. Make sure that the outlying observations were entered correctly.

    You now have a choice:

    1. You can delete the outlying observations in a listwise fashion. This should be done only if the number of outlying variables is less than 10% of the overall distribution. This is the least preferable choice.

    2. You can conduct a logarithmic transformation on the outcome variable. Doing this will normalize the distribution so that you can run the analysis using parametric statistics. The unstandardized beta coefficients, standard errors, and standardized beta coefficients are not interpretable, but the significance of the associations between the predictor variables and the transformed outcome can yield some inferential evidence.

    3. You can recode the continuous outcome variable into a lower level scale of measurement such as ordinal or categorical and run non-parametric statistics to seek out any associations. Of course, you are losing the precision and accuracy of continuous-level measurement and introducing measurement error into the outcome variable, but you will still be able to run inferential statistics.

    4. You can use non-parametric statistics without changing the skewed variable at all. That is one of the primary benefits of non-parametric statistics: They are robust to violations of normality and homogeneity of variance. Instead of interpreting means and standard deviations, you will interpret medians and interquartile ranges with non-parametric statistics. 

    Click on the Statistics button to learn more.
  • Published on

    Statistical tests

    Hypothesis Testing Research Question Statistics

    Statistical tests are used to answer research questions

    It's not about the statistics, it's about the question.

    In my experience, statistics is a cognitive dissonance-inducing mathematical science and no one tends to recall their personal and professional statistical experiences with much zeal. It's as if there is an automatic recoil when the topic of statistics enters the discussion and planning of a research study. The literature has posited that statistics are intimidating and nebulous because many people do not possess the necessary competencies and experience with statistics and also people do not understand the lexicon of the science.

    The most important thing to remember about applied statistics, despite their prevalence, relevance, and utility in everyday life, is that they are tools that human beings use to communicate the results of data analysis. Hypothesis testing is employed in empirical research so that researchers can present their findings in a relative context that is interpretable and applicable in other research and applied environments.

    Statistics are useful ONLY when they are used to answer useful, appropriate, answerable, relevant, and valid research questions that are grounded in the empirical literature.