Statistical Consultation Line: (865) 742-7731
Accredited Professional Statistician For Hire
  • Contact Form

Causality in Statistical Power: Isomorphic Properties of Measurement, Research Design, Effect Size, and Sample Size

3/18/2016

1 Comment

 

Newest publication from Dr. Eric Heidel, creator of Research Engineer

Directly related to Statistical Power engine of Research Engineer

My newest published article in Scientifica is now available for download online and on the Research Engineer website. The creation of the Statistical Power engine of Research Engineer led me to write the article. Click on the Download Article button below to download a .pdf directly from the website or click on the Statistical Power button to be taken to the aforementioned engine. Many thanks and regards to everyone that uses Research Engineer! -EH

Download Article
Statistical Power
Research Engineer Home Page
Scale, LLC
1 Comment

Research Engineer makes applied research and statistics easier

10/22/2015

0 Comments

 

Research Engineer is designed to get you to the correct research question, research design, sample size, database, and statistical test

Based on your decisions to the questions presented, you will get to right place

A few words on what I'm doing on here. I am a biostatistician, methodologist, psychometrician, and counselor. Everyday, the incredibly intelligent people I work with including physicians, residents, fellows, staff, and faculty feel anxiety when it comes to statistics and research. Research has shown that statistics can induce cognitive dissonance in an individual due to limited experiences and competencies. The collective unconscious has sequestered statistics and research into a dark corner and that's scary.

Research and statistics are the methods by which we, as scientists, analyze, synthesize, and evaluate our research findings in a manner that can be generalized to the appropriate audience. If our methods for communicating research findings causes cognitive dissonance, just because it relates to research and statistics, then how can one ever really be able to generalize the clinical literature and integrate it into clinical practice?

After seven years of being the one to induce cognitive dissonance in others related to research and statistics, I decided to make a useful tool for students and researchers that could alleviate some of the feelings of anxiety associated with research and statistics. I built Research Engineer.

Research Engineer is designed to get you to the correct research question, research design, sample size, database, statistical test, evidence-based medicine intervention, diagnostic calculation, epidemiological calculation, variables, surveys, psychometrics, and educational framework to answer your current question (and future questions). 

I am trying to bring research and statistics out of the collective unconscious and into the conscious mind where it can be effectively communicated among researchers, scientists, and students by creating this decision engine. It is easy to get to the correct research or statistical component, just answer the questions that I present you in the webpages and click on the buttons with your answer in them. Also, the step-by-step methods for conducting and interpreting each statistical test in SPSS are presented on their respective webpages. 

You can also contact me via phone, social media, and email at any time in you have questions. If you need some help conducting statistics for a research project, I have eight years of experience across thousands of individual projects and I would love to help you on your study.  We can negotiate prices if you are an undergraduate or graduate researcher. 

In conclusion, Research Engineer makes choosing research methods and statistical tests MUCH EASIER. Just answer the questions embedded in the various decision engines and get to the correct method or test, EVERY TIME.

Thanks for your continued support, dear friends and colleagues. And many thanks and salutations to the individuals that use Research Engineer. I am honored and humbled to have this great opportunity to create a very useful and unique website. You all are the ones that make it shine!

​Sincerely,

R. Eric Heidel, Ph.D.
Assistant Professor of Biostatistics
​Affiliate Professor of Biomedical Engineering
Department of Surgery
Office of Medical Education, Research, and Development
University of Tennessee Graduate School of Medicine
Owner and Operator, Scale, LLC

Scale, LLC
0 Comments

Within-subjects designs increase statistical power

10/4/2015

1 Comment

 

Within-subjects designs increase statistical power

Each participant serves as their own control in within-subjects designs

Within-subjects designs increase statistical power. because participants serve as their own control. Between-subjects designs necessitate more observations of the outcome to be able to effectively compare independent groups on an outcome. Multivariate analyses further decrease statistical power in that many more observations of the outcome to detect significant effects. At least 20 -40 more observations of the outcome have to collected per variable entered into a simultaneous of hierarchial regression model in order to meet statistical power when trying to account for demographic, etiological, clinical, and confounding effects.

Within-subjects designs, when coupled with with continuous outcomes, large effect sizes, limited variance in the outcome and a large sample size, greatly increase statistical power. Small effect sizes are also easier to detect using within-subjects statistics because participants serve as their own control. Within-subjects design also provide more statistical power when small sample sizes are used.    

Scale, LLC
1 Comment

Categorical measurement caveats

4/1/2015

0 Comments

 

Effects of categorical measurement

Decrease statistical power and increase sample size

Categorical variables are very prevalent in medicine. Measures like presence of comorbidities, mortality, and test results are categorical in nature. Here are some general caveats associated with categorical measurement and sample size:  

1. Categorical outcomes will always DECREASE statistical power and INCREASE the needed sample size. This is due to the lack of precision and accuracy in categorical measurement.

2. The underlying algebra associated with calculating 95% confidence intervals of odds ratios and relative risk is 100% dependent upon the sample size. With smaller sample sizes, by default, wider and less precise 95% confidence intervals will be found. If one of the cells of a cross-tabulation table has fewer observations that the other cells, then the 95% confidence interval will be wider and potentially not truly interpretable. A 95% confidence interval will become narrower or more precise only with larger sample sizes.  

3. When using categorical variables for diagnostic testing purposes, larger samples sizes will be needed to calculate precise measures of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV). With smaller sample sizes in diagnostic studies, a change in one or two observations can have drastic effects on the diagnostic values.

This is especially true when there is a subjective rating used for purposes of diagnosing someone as "positive" or "negative" for a given disease state (radiologist reading an X-ray). Inter-rater reliability coefficients such as Kappa or ICC should be employed to ensure consistency and reliability among subsequent ratings and raters. Sensitivity, specificity, and PPV will be affected by inter-rater reliability. Receiver Operator Characteristic (ROC) curves can be used to find a given value where sensitivity and specificity of a test is maximized. ROC curves can also be used to compare the area under the curve (AUC) between several diagnostic tests at the same time so that the best can be chosen.  

4. For each predictor categorical parameter (or variable) that you want to include in a multivariate model, you have to increase your sample size by at least 20-40 observations of the outcome. This due to the limited precision, accuracy, and statistical power associated with categorical measurement. Researchers HAVE to collect more observations in order to detect any potential significant multivariate associations.  

In the case that a polychotomous variable is to be used in a model, create (a-1), where a is the number of categories, dichotomous variables with "0" as not being that category and "1" as being that category. For each level, 20-40 more observations of the outcome will be needed to have enough statistical power to detect differences amongst the multiple groups.        

Scale, LLC
0 Comments

Small sample sizes, Type II errors, and empirical reasoning

11/18/2014

0 Comments

 

Small sample sizes can lead to Type II errors

Significant effects may not be able to be detected

In instances where a phenomenon or outcome is less prevalent in the population, scientists are forced to work small sample sizes. It is just the nature of the science, and the phenomenon or outcome.

1. When working with smaller sample sizes, adequate statistical power (and therefore statistical significance) is VERY hard to achieve.

2. There is limited precision and accuracy when using categorical or ordinal outcomes, which can further decreases statistical power.

3. When measuring for small effect sizes, small sample sizes cannot provide enough variance in the outcome to detect clinically meaningful, but small effects. This REALLY decreases your statistical power since inferential statistics depend upon variance in the mathematical sense.

With this being said, remember to interpret the p-values yielded from RCT level studies with small sample sizes in the context of the aforementioned points. If a treatment effect does not obtain statistical significance, but appears to be CLINICALLY SIGNIFICANT with a p-value approaching significance (Type II error), then perhaps more credence can be found in the effect.

If researchers run bivariate tests on 30 different outcomes with less than 20 observations and claim significance without a Bonferroni adjustment, throw the article out.

Scale, LLC
0 Comments

Feasible research questions are answerable

10/31/2014

0 Comments

 

Feasible research in terms of scope, time, resources, and expertise

Changing the face of medicine versus completing a research study

I have conducted thousands of statistical consultations over the years and have worked with many novice resident researchers over that time. One cannot help but admire the spirit, energy, and motivation of young people wanting to make an impact on medicine through research. I enjoy the zeal and drive of bright people wanting to be physicians and researchers. This is a good thing!

That being said, I spend a lot of my time with novice researchers using deductive reasoning to hone down their research questions into something tangible and feasible. They come into the office with an idea that will change medicine forever and we will be cruising around the Caribbean in a year! This has never been researched before!  No one has ever done this before! Trust me, I want all of these proclamations to be true and I also want to change the face of medicine. Yet, most times it just not feasible to do so given the time, resources, participants, competencies and environment associated with the study.

I focus on a few primary areas when it comes to feasible research questions with my consultees:

1. Participant pool - Are there enough participants available in the immediate clinical or empirical environment to achieve adequate statistical power for inferential analyses? How will you recruit the participants? What are your inclusion and exclusion criteria? Inclusion and exclusion criteria may need to be modified to increase sample size.

2. Effect size - Small effect sizes require large sample sizes.    

3. Research design - Retrospective designs are always more feasible because the data already exists.

4. Communication - Research never occurs in isolation. Researchers should communicate and collaborate with their peers regarding their research projects. Attendings and academic physicians can give you ideas on how to feasibly conduct your research.

5. Time - What is the time frame for the study from inception to publication? How much time do you have to set aside for the research study? Does the completion of your research coincide with abstract deadlines of interest?

6. Power analysis - Conduct an a priori power anlaysis based on an evidence-based measure of effect to see if the study is feasible in regards to sample size needed to achieve power.

Scale, LLC
0 Comments

Values needed for sample size calculations

10/8/2014

0 Comments

 

Evidence-based measures of effect

Use the empirical literature to your advantage

One of the most important things you can do when designing your study is to conduct an a priori power analysis. Doing so will tell you how many people that you will need in your sample size to detect the effect size or treatment effect in your study.

Without an a priori calculation, you could frivolously waste months or years of your life conducting a study only to find out that you only needed 100 in each group to achieve significance. Or, with the inverse, you conduct a study with only 50 patients and find out in a post hoc fashion that you would have needed 10,000 to prove your effect!  

If you are using Research Engineer and G*Power to run your analyses, here are the things you will need:

1. An evidence-based measure of effect from the literature is the first thing you should seek out. Find a study that is theoretically, conceptually, or clinically similar to your own. Try to find a study that uses the same outcome you plan to use in your study.  

2. Use the means, standard deviations, and proportions from these published studies as evidence-based measures of effect size to calculate how large of a sample size you will need. These values will be reported in body of the results section or in tables within the manuscript. It shows more empirical rigor on your part if you conduct an a priori power analysis based on a well-known study in the field.

3. Plug these values into G*Power using the steps published on the sample size page to find out how many people you will need to collect for your study.

Scale, LLC
0 Comments

Non-parametric statistics and small sample sizes

10/5/2014

0 Comments

 

Non-parametric statistics are robust to small sample sizes

The right way to conduct statistics

Mark Twain said it best, "There are lies, damn lies, and statistics." Statistics can be misleading from both the standpoint of the person conducting the statistics and the person that is interpreting the analyses. Many between-subjects studies have small sample sizes (n < 20) and statistical assumptions for parametric statistics cannot be met.

For basic researchers that operate day in and day out with small sample sizes, the answer is to use non-parametric statistics. Non-parametric statistical tests such as the Mann-Whitney U, Kruskal-Wallis, Wilcoxon, and Friedman's ANOVA are robust to violations of statistical assumptions and skewed distributions. These tests can yield interpretable medians, interquartile ranges, and p-values.

Non-parametric statistics are also useful in the social sciences due to the inherent measurement error associated with assessing human behaviors, thoughts, feelings, intelligence, and emotional states. The underlying algebra associated with psychometrics relies on intercorrelations amongst constructs or items.  Correlations can easily be skewed by outlying observations and measurement error.  Therefore, in concordance with mathematical and empirical reasoning, non-parametric statistics should be used often for between-subjects comparisons of surveys, instruments, and psychological measures.

Scale, LLC
0 Comments

G*Power for the masses

9/26/2014

0 Comments

 

G*Power is a necessary tool for every researcher's toolkit

Easy statistical power and sample size calculations

I'm trying to run an online business so I'm fully Google-integrated. I see that there many search queries of different derivations related to sample size calculation as it relates to behind-the-scenes tracking measures.

There is an open-source tool available to EVERYONE that allows you to calculate your own a priori and post hoc power analyses. It is called G*Power and as your personal statistical consultant, I highly suggest you go to the following web address and download Version 3.0 to your respective device:

http://www.gpower.hhu.de/en.html    

The researchers that developed this program have made a great contribution to science. It is truly a great and FREE program that can run a litany of different power analyses. You can find out in minutes how large of a sample size that you need, given that you have an idea of the effect size that you are attempting to detect in your study.

Use means, proportions, and variance measures from published studies in your field to have the most empirically rigorous hypothesized effect. Enter these values into G*Power and the adjust the variance and magnitude of the effect size to see how the required sample size changes.   

Click on the Sample Size button to access the methods of conducting and interpreting sample size calculations for ten different statistical tests.

Sample Size
Scale, LLC
0 Comments

Preliminary statistical consultation

9/26/2014

0 Comments

 

Support your local statistician!

Seek out methodological and statistical consultation

If you have access to a statistical consultants or statisticians within your empirical or clinical environment, seek out their services in the preliminary phases of planning your study. Here is a list of things that I do for residents, fellows, faculty, physicians, pharmacists, nurses, and staff at an academic regional medical campus:

1. Sample Size - I conduct sample size calculations for at least of 80-85% of my first-time clients. They often want to know how many people they need to reach a significant p-value. We work through the process of acquiring an evidence-based measure of effect that reflects what their research question is trying to answer.

It feels good knowing that you have a good chance of detecting significance with a small sample size. Also, it is good to find out that you have to collect A LOT more observations than you thought you would. Post hoc power analyses should be run for any non-significant main effects that may be considered Type II errors (limited or small sample sizes).

2. Statistical analysis - Real biostatistical scientists and statisticians will conduct your statistical analyses in an objective and expeditious manner to help you answer your research questions. Please help them understand what your research question is and what research design you want to use to answer it to the best of your abilities. They will be able to help you choose the correct statistic given that you can tell them the scale of measurement for your primary outcome and what type of design (between-subjects, within-subjects, correlational, mixed, or multivariate) you want to use to answer your question. It is also important to know WHO or WHAT you want to include in your sample in terms of inclusion and exclusion criteria. Finally, know your content area. We may not know your knowledge/philosophical base and need to understand the entire picture, as much as you can tell us.

3. Database management - Go ahead and let us build your database in a basic Excel spreadsheet and send an accompanying code book in Word so that we are all on the same page. It helps us all know what is going on, what variables are being collected, what they mean, how they are measured, and how the analysis will work. Share it with all members of the research team. Use the code book when entering your data. Tell the rest of us if you make changes to the code book or database. These simple tasks and communicative efforts can mean the difference between your statistics being run in five minutes versus five weeks.  SERIOUSLY.

4. Write-up of findings for publication - We will give you an annotated write-up of your findings with statistical outputs and give you basic and unbiased interpretations of the statistical results of your study. We can help you write up the statistical methods and results sections of your abstracts and manuscripts. We can even help you design tables and graphs that will make your study findings more aesthetically and visually appealing to your audience.

When it comes to authorship, if you feel that your statistical professional's contribution to the design, execution, and interpretation of your study warrants authorship, offer it to them. They will greatly appreciate it! However, YOU SHOULD NEVER BE REQUIRED TO GIVE US AUTHORSHIP JUST BECAUSE WE RAN YOUR STATISTICS FOR YOU.  IT IS UNETHICAL FOR US TO REQUIRE AUTHORSHIP FOR DOING OUR JOB. THAT IS, IF OUR JOB IS TO RUN STATISTICS IN YOUR EMPIRICAL OR CLINICAL ENVIRONMENT.          

Scale, LLC
0 Comments
<<Previous

    Archives

    March 2016
    January 2016
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014

    Author

    Eric Heidel, Ph.D. is Owner and Operator of Scalë, LLC.

    Categories

    All
    95% Confidence Interval
    Absolute Risk Reduction
    Accuracy
    Acquiring Clinical Evidence
    Adjusted Odds Ratio
    Affordable Care Act
    Alpha Value
    ANCOVA Test
    ANOVA Test
    Applying Clinical Evidence
    Appraisal Of The Literature
    Appraising Clinical Evidence
    A Priori
    Area Under The Curve
    Asking Clinical Questions
    Assessing Clinical Practice
    AUC
    Basic Science
    Beta Value
    Between-subjects
    Biserial
    Blinding
    Bloom's Taxonomy
    Bonferroni
    Boolean Operators
    Calculator
    Case-control Design
    Case Series
    Categorical
    Causal Effects
    Chi-square
    Chi-square Assumption
    Chi-square Goodness-of-fit
    Classical Test Theory
    Clinical Pathways
    Clustered Random Sampling
    Cochran-Mantel-Haenszel
    Cochran's Q Test
    Coefficient Of Determination
    Cognitive Dissonance
    Cohort
    Comparative Effectiveness Research
    Comparator
    Concurrent Validity
    Confidence Interval
    Confirmatory Factor Analysis
    Construct Specification
    Construct Validity
    Continuous
    Control Event Rate
    Convenience Sampling Method
    Convergent Validity
    Copyright
    Correlations
    Count Variables
    Cox Regression
    Cronbach's Alpha
    Cross-sectional
    Curriculum Vitae
    Database Management
    Diagnostic Testing
    EBM
    Education
    Effect Size
    Empirical Literature
    Epidemiology
    Equivalency Trial
    Eric Heidel
    Evidence-based Medicine
    Exclusion Criteria
    Experimental Designs
    Experimental Event Rate
    Facebook
    Factorial ANOVA
    Feasible Research Questions
    FINER
    Fisher's Exact Tests
    Friedman's ANOVA
    Generalized Estimating Equations (GEE)
    "gold Standard" Outcome
    G*Power
    Guidelines For Authors
    Hazard Ratio
    Hierarchical Regression
    Homogeneity Of Variance
    Hypothesis Testing
    ICC
    Incidence
    Inclusion Criteria
    Independence Of Observations Assumption
    Independent Samples T-test
    Intention-to-treat
    Internal Consistency Reliability
    Interquartile Range
    Inter-rater Reliability
    Interval Variables
    Intervention
    Intraclass Correlation Coefficient
    Isomorphism
    Item Response Theory
    Kaplan-Meier Curve
    Kappa Statistic
    KR-20
    Kruskal-Wallis
    Kurtosis
    Levene's Test
    Likert Scales
    Linearity
    Listwise Deletion
    Logarithmic Transformations
    Logistic Regression
    Log-Rank Test
    Longitudinal Data
    MANCOVA
    Mann-Whitney U
    MANOVA
    Mass Emails In Survey Research
    Math
    Mauchly's Test
    McNemar's Test
    Mean
    Measurement
    Median
    Medicine
    Merging Databases
    Missing Data
    Mode
    Multinomial Logistic Regression
    Multiple Regression
    Multivariate Statistics
    Negative Binomial Regression
    Negative Predictive Value
    Nominal Variables
    Nonequivalent Control Group Design
    Non-inferiority
    Non-inferiority Trial
    Non-parametric Statistics
    Non-probability Sampling
    Normality
    Normality Of Difference Scores
    Normal Probability Plot
    Novel Research Question
    Number Needed To Treat
    Observational Research
    Odds Ratio With 95% CI
    One-sample Median Tests
    One-sample T-test
    One-sided Hypothesis
    One-Way Random
    Operationalization
    Ordinal
    Outcome
    Outliers
    Parametric Statistics
    Pearson's R
    Ph.D.
    Phi Coefficient
    PICO
    Pilot Study
    Point Biserial
    Poisson Regression
    Population
    Positive Predictive Value
    Post Hoc
    Post-positivism
    PPACA
    PPV
    Precision
    Predictive Validity
    Prevalence
    Principal Components Analysis
    Probability Sampling
    Propensity Score Matching
    Proportion
    Proportional Odds Regression
    Prospective Cohort
    Psychometrics
    Psychometric Tests
    Publication
    Publication Bias
    Purposive Sampling
    P-value
    Random Assignment
    Randomized Controlled Trial
    Random Selection
    Rank Biserial
    Ratio Variables
    Receiver Operator Characteristic
    Regression
    Regression Analysis
    Relative Risk
    Relevant Research Question
    Reliability
    Repeated-measures ANOVA
    Repeated-measures T-test
    Research
    Research Design
    Research Engineer
    Research Journal
    Research Question
    Residual Analysis
    Retrospective Cohort
    ROC Curve
    Sample Size
    Sampling
    Sampling Error
    Sampling Method
    Scales Of Measurement
    Science
    Search Engine
    Search Query
    Sensitivity
    Simple Random Sampling
    Sitemap
    Skewness
    Social Science
    Spearman-Brown
    Spearman's Rho
    Specificity
    Specificity In Literature Searching
    Sphericity Assumption
    Split-half Reliability
    SPSS
    Standard Deviation
    Standards Of Care
    Statistical Analysis
    Statistical Assumptions
    Statistical Consultation
    Statistical Power
    Statistical Power Analysis
    Statistical-power-test
    Statistician
    Statistics
    Stratified Random Sampling
    Survey
    Survey Construct Specification
    Survey Methods
    Systematic Review
    Test-Retest Reliability
    Twitter
    Two-sided Hypothesis
    Two-Way Mixed
    Two-Way Random
    Type I Error
    Type II Error
    Unadjusted Odds Ratio
    Validity
    Variables
    Variance
    Wilcoxon
    Within-subjects
    YouTube


    Contact Form

Contact Dr. Eric Heidel
consultation@scalelive.com
(865) 742-7731

Copyright © 2022 Scalë. All Rights Reserved. Patent Pending.