Tags

  • Published on

    Causality in Statistical Power: Isomorphic Properties of Measurement, Research Design, Effect Size, and Sample Size

    Chi-square Effect Size Isomorphism Kruskal-Wallis Kurtosis Mann-Whitney U Measurement Non-parametric Statistics Normality Odds Ratio With 95% CI Relative Risk Research Design Research Question Sample Size Skewness Statistical Assumptions Statistical Power Statistical Power Analysis Statistics Survey Wilcoxon

    Newest publication from Dr. Eric Heidel, creator of Research Engineer

    Directly related to Statistical Power engine of Research Engineer

    My newest published article in Scientifica is now available for download online and on the Research Engineer website. The creation of the Statistical Power engine of Research Engineer led me to write the article. Click on the Download Article button below to download a .pdf directly from the website or click on the Statistical Power button to be taken to the aforementioned engine. Many thanks and regards to everyone that uses Research Engineer! -EH
  • Published on

    Research Engineer makes applied research and statistics easier

    Chi-square Database Management Diagnostic Testing Effect Size Epidemiology Evidence-based Medicine Kruskal-Wallis Kurtosis Mann-Whitney U Non-parametric Statistics Normality Odds Ratio With 95% CI Psychometrics Relative Risk Research Research Design Research Question Sample Size Skewness Statistical Assumptions Statistical Power Analysis Statistics Survey Variables Wilcoxon

    Research Engineer is designed to get you to the correct research question, research design, sample size, database, and statistical test

    Based on your decisions to the questions presented, you will get to right place

    A few words on what I'm doing on here. I am a biostatistician, methodologist, psychometrician, and counselor. Everyday, the incredibly intelligent people I work with including physicians, residents, fellows, staff, and faculty feel anxiety when it comes to statistics and research. Research has shown that statistics can induce cognitive dissonance in an individual due to limited experiences and competencies. The collective unconscious has sequestered statistics and research into a dark corner and that's scary.

    Research and statistics are the methods by which we, as scientists, analyze, synthesize, and evaluate our research findings in a manner that can be generalized to the appropriate audience. If our methods for communicating research findings causes cognitive dissonance, just because it relates to research and statistics, then how can one ever really be able to generalize the clinical literature and integrate it into clinical practice?

    After seven years of being the one to induce cognitive dissonance in others related to research and statistics, I decided to make a useful tool for students and researchers that could alleviate some of the feelings of anxiety associated with research and statistics. I built Research Engineer.

    Research Engineer is designed to get you to the correct research question, research design, sample size, database, statistical test, evidence-based medicine intervention, diagnostic calculation, epidemiological calculation, variables, surveys, psychometrics, and educational framework to answer your current question (and future questions). 

    I am trying to bring research and statistics out of the collective unconscious and into the conscious mind where it can be effectively communicated among researchers, scientists, and students by creating this decision engine. It is easy to get to the correct research or statistical component, just answer the questions that I present you in the webpages and click on the buttons with your answer in them. Also, the step-by-step methods for conducting and interpreting each statistical test in SPSS are presented on their respective webpages. 

    You can also contact me via phone, social media, and email at any time in you have questions. If you need some help conducting statistics for a research project, I have eight years of experience across thousands of individual projects and I would love to help you on your study.  We can negotiate prices if you are an undergraduate or graduate researcher. 

    In conclusion, Research Engineer makes choosing research methods and statistical tests MUCH EASIER. Just answer the questions embedded in the various decision engines and get to the correct method or test, EVERY TIME.

    Thanks for your continued support, dear friends and colleagues. And many thanks and salutations to the individuals that use Research Engineer. I am honored and humbled to have this great opportunity to create a very useful and unique website. You all are the ones that make it shine!

    ​Sincerely,

    R. Eric Heidel, Ph.D.
    Assistant Professor of Biostatistics
    ​Affiliate Professor of Biomedical Engineering
    Department of Surgery
    Office of Medical Education, Research, and Development
    University of Tennessee Graduate School of Medicine
    Owner and Operator, Scale, LLC
  • Published on

    Within-subjects designs increase statistical power

    Chi-square Effect Size Kruskal-Wallis Kurtosis Mann-Whitney U Non-parametric Statistics Normality Odds Ratio With 95% CI Outcome Relative Risk Research Design Research Question Sample Size Skewness Statistical Assumptions Statistical Power Statistical Power Analysis Statistics Survey Wilcoxon

    Within-subjects designs increase statistical power

    Each participant serves as their own control in within-subjects designs

    Within-subjects designs increase statistical power. because participants serve as their own control. Between-subjects designs necessitate more observations of the outcome to be able to effectively compare independent groups on an outcome. Multivariate analyses further decrease statistical power in that many more observations of the outcome to detect significant effects. At least 20 -40 more observations of the outcome have to collected per variable entered into a simultaneous of hierarchial regression model in order to meet statistical power when trying to account for demographic, etiological, clinical, and confounding effects.

    Within-subjects designs, when coupled with with continuous outcomes, large effect sizes, limited variance in the outcome and a large sample size, greatly increase statistical power. Small effect sizes are also easier to detect using within-subjects statistics because participants serve as their own control. Within-subjects design also provide more statistical power when small sample sizes are used.    
  • Published on

    Categorical measurement caveats

    95% Confidence Interval Categorical Diagnostic Testing Inter-rater Reliability Intraclass Correlation Coefficient Kappa Statistic Multivariate Statistics Negative Predictive Value Odds Ratio With 95% CI Positive Predictive Value Relative Risk Sample Size Sensitivity Specificity Statistical-power-test

    Effects of categorical measurement

    Decrease statistical power and increase sample size

    Categorical variables are very prevalent in medicine. Measures like presence of comorbidities, mortality, and test results are categorical in nature. Here are some general caveats associated with categorical measurement and sample size:  

    1. Categorical outcomes will always DECREASE statistical power and INCREASE the needed sample size. This is due to the lack of precision and accuracy in categorical measurement.

    2. The underlying algebra associated with calculating 95% confidence intervals of odds ratios and relative risk is 100% dependent upon the sample size. With smaller sample sizes, by default, wider and less precise 95% confidence intervals will be found. If one of the cells of a cross-tabulation table has fewer observations that the other cells, then the 95% confidence interval will be wider and potentially not truly interpretable. A 95% confidence interval will become narrower or more precise only with larger sample sizes.  

    3. When using categorical variables for diagnostic testing purposes, larger samples sizes will be needed to calculate precise measures of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV). With smaller sample sizes in diagnostic studies, a change in one or two observations can have drastic effects on the diagnostic values.

    This is especially true when there is a subjective rating used for purposes of diagnosing someone as "positive" or "negative" for a given disease state (radiologist reading an X-ray). Inter-rater reliability coefficients such as Kappa or ICC should be employed to ensure consistency and reliability among subsequent ratings and raters. Sensitivity, specificity, and PPV will be affected by inter-rater reliability. Receiver Operator Characteristic (ROC) curves can be used to find a given value where sensitivity and specificity of a test is maximized. ROC curves can also be used to compare the area under the curve (AUC) between several diagnostic tests at the same time so that the best can be chosen.  

    4. For each predictor categorical parameter (or variable) that you want to include in a multivariate model, you have to increase your sample size by at least 20-40 observations of the outcome. This due to the limited precision, accuracy, and statistical power associated with categorical measurement. Researchers HAVE to collect more observations in order to detect any potential significant multivariate associations.  

    In the case that a polychotomous variable is to be used in a model, create (a-1), where a is the number of categories, dichotomous variables with "0" as not being that category and "1" as being that category. For each level, 20-40 more observations of the outcome will be needed to have enough statistical power to detect differences amongst the multiple groups.        
  • Published on

    Small sample sizes, Type II errors, and empirical reasoning

    Accuracy Bonferroni Effect Size P-value Precision Sample Size Statistical-power-test Type II Error

    Small sample sizes can lead to Type II errors

    Significant effects may not be able to be detected

    In instances where a phenomenon or outcome is less prevalent in the population, scientists are forced to work small sample sizes. It is just the nature of the science, and the phenomenon or outcome.

    1. When working with smaller sample sizes, adequate statistical power (and therefore statistical significance) is VERY hard to achieve.

    2. There is limited precision and accuracy when using categorical or ordinal outcomes, which can further decreases statistical power.

    3. When measuring for small effect sizes, small sample sizes cannot provide enough variance in the outcome to detect clinically meaningful, but small effects. This REALLY decreases your statistical power since inferential statistics depend upon variance in the mathematical sense.

    With this being said, remember to interpret the p-values yielded from RCT level studies with small sample sizes in the context of the aforementioned points. If a treatment effect does not obtain statistical significance, but appears to be CLINICALLY SIGNIFICANT with a p-value approaching significance (Type II error), then perhaps more credence can be found in the effect.

    If researchers run bivariate tests on 30 different outcomes with less than 20 observations and claim significance without a Bonferroni adjustment, throw the article out.
  • Published on

    Feasible research questions are answerable

    Effect Size Feasible Research Questions Research Design Research Question Sample Size Statistical Power Analysis Statistical-power-test

    Feasible research in terms of scope, time, resources, and expertise

    Changing the face of medicine versus completing a research study

    I have conducted thousands of statistical consultations over the years and have worked with many novice resident researchers over that time. One cannot help but admire the spirit, energy, and motivation of young people wanting to make an impact on medicine through research. I enjoy the zeal and drive of bright people wanting to be physicians and researchers. This is a good thing!

    That being said, I spend a lot of my time with novice researchers using deductive reasoning to hone down their research questions into something tangible and feasible. They come into the office with an idea that will change medicine forever and we will be cruising around the Caribbean in a year! This has never been researched before!  No one has ever done this before! Trust me, I want all of these proclamations to be true and I also want to change the face of medicine. Yet, most times it just not feasible to do so given the time, resources, participants, competencies and environment associated with the study.

    I focus on a few primary areas when it comes to feasible research questions with my consultees:

    1. Participant pool - Are there enough participants available in the immediate clinical or empirical environment to achieve adequate statistical power for inferential analyses? How will you recruit the participants? What are your inclusion and exclusion criteria? Inclusion and exclusion criteria may need to be modified to increase sample size.

    2. Effect size - Small effect sizes require large sample sizes.    

    3. Research design - Retrospective designs are always more feasible because the data already exists.

    4. Communication - Research never occurs in isolation. Researchers should communicate and collaborate with their peers regarding their research projects. Attendings and academic physicians can give you ideas on how to feasibly conduct your research.

    5. Time - What is the time frame for the study from inception to publication? How much time do you have to set aside for the research study? Does the completion of your research coincide with abstract deadlines of interest?

    6. Power analysis - Conduct an a priori power anlaysis based on an evidence-based measure of effect to see if the study is feasible in regards to sample size needed to achieve power.