Tags

  • Published on

    Values needed for sample size calculations

    A Priori Effect Size G*Power Mean Proportion Research Engineer Sample Size Standard Deviation Statistical Power Analysis

    Evidence-based measures of effect

    Use the empirical literature to your advantage

    One of the most important things you can do when designing your study is to conduct an a priori power analysis. Doing so will tell you how many people that you will need in your sample size to detect the effect size or treatment effect in your study.

    Without an a priori calculation, you could frivolously waste months or years of your life conducting a study only to find out that you only needed 100 in each group to achieve significance. Or, with the inverse, you conduct a study with only 50 patients and find out in a post hoc fashion that you would have needed 10,000 to prove your effect!  

    If you are using Research Engineer and G*Power to run your analyses, here are the things you will need:

    1. An evidence-based measure of effect from the literature is the first thing you should seek out. Find a study that is theoretically, conceptually, or clinically similar to your own. Try to find a study that uses the same outcome you plan to use in your study.  

    2. Use the means, standard deviations, and proportions from these published studies as evidence-based measures of effect size to calculate how large of a sample size you will need. These values will be reported in body of the results section or in tables within the manuscript. It shows more empirical rigor on your part if you conduct an a priori power analysis based on a well-known study in the field.

    3. Plug these values into G*Power using the steps published on the sample size page to find out how many people you will need to collect for your study.
  • Published on

    Non-parametric statistics and small sample sizes

    Friedman's ANOVA Kruskal-Wallis Mann-Whitney U Non-parametric Statistics Psychometric Tests Sample Size Wilcoxon

    Non-parametric statistics are robust to small sample sizes

    The right way to conduct statistics

    Mark Twain said it best, "There are lies, damn lies, and statistics." Statistics can be misleading from both the standpoint of the person conducting the statistics and the person that is interpreting the analyses. Many between-subjects studies have small sample sizes (n < 20) and statistical assumptions for parametric statistics cannot be met.

    For basic researchers that operate day in and day out with small sample sizes, the answer is to use non-parametric statistics. Non-parametric statistical tests such as the Mann-Whitney U, Kruskal-Wallis, Wilcoxon, and Friedman's ANOVA are robust to violations of statistical assumptions and skewed distributions. These tests can yield interpretable medians, interquartile ranges, and p-values.

    Non-parametric statistics are also useful in the social sciences due to the inherent measurement error associated with assessing human behaviors, thoughts, feelings, intelligence, and emotional states. The underlying algebra associated with psychometrics relies on intercorrelations amongst constructs or items.  Correlations can easily be skewed by outlying observations and measurement error.  Therefore, in concordance with mathematical and empirical reasoning, non-parametric statistics should be used often for between-subjects comparisons of surveys, instruments, and psychological measures.
  • Published on

    G*Power for the masses

    Effect Size G*Power Sample Size Sampling Statistical Power Analysis

    G*Power is a necessary tool for every researcher's toolkit

    Easy statistical power and sample size calculations

    I'm trying to run an online business so I'm fully Google-integrated. I see that there many search queries of different derivations related to sample size calculation as it relates to behind-the-scenes tracking measures.

    There is an open-source tool available to EVERYONE that allows you to calculate your own a priori and post hoc power analyses. It is called G*Power and as your personal statistical consultant, I highly suggest you go to the following web address and download Version 3.0 to your respective device:

    http://www.gpower.hhu.de/en.html    

    The researchers that developed this program have made a great contribution to science. It is truly a great and FREE program that can run a litany of different power analyses. You can find out in minutes how large of a sample size that you need, given that you have an idea of the effect size that you are attempting to detect in your study.

    Use means, proportions, and variance measures from published studies in your field to have the most empirically rigorous hypothesized effect. Enter these values into G*Power and the adjust the variance and magnitude of the effect size to see how the required sample size changes.   

    Click on the Sample Size button to access the methods of conducting and interpreting sample size calculations for ten different statistical tests.
  • Published on

    Preliminary statistical consultation

    Database Management Research Engineer Sample Size Statistical Analysis Statistical Consultation Statistician Statistics

    Support your local statistician!

    Seek out methodological and statistical consultation

    If you have access to a statistical consultants or statisticians within your empirical or clinical environment, seek out their services in the preliminary phases of planning your study. Here is a list of things that I do for residents, fellows, faculty, physicians, pharmacists, nurses, and staff at an academic regional medical campus:

    1. Sample Size - I conduct sample size calculations for at least of 80-85% of my first-time clients. They often want to know how many people they need to reach a significant p-value. We work through the process of acquiring an evidence-based measure of effect that reflects what their research question is trying to answer.

    It feels good knowing that you have a good chance of detecting significance with a small sample size. Also, it is good to find out that you have to collect A LOT more observations than you thought you would. Post hoc power analyses should be run for any non-significant main effects that may be considered Type II errors (limited or small sample sizes).

    2. Statistical analysis - Real biostatistical scientists and statisticians will conduct your statistical analyses in an objective and expeditious manner to help you answer your research questions. Please help them understand what your research question is and what research design you want to use to answer it to the best of your abilities. They will be able to help you choose the correct statistic given that you can tell them the scale of measurement for your primary outcome and what type of design (between-subjects, within-subjects, correlational, mixed, or multivariate) you want to use to answer your question. It is also important to know WHO or WHAT you want to include in your sample in terms of inclusion and exclusion criteria. Finally, know your content area. We may not know your knowledge/philosophical base and need to understand the entire picture, as much as you can tell us.

    3. Database management - Go ahead and let us build your database in a basic Excel spreadsheet and send an accompanying code book in Word so that we are all on the same page. It helps us all know what is going on, what variables are being collected, what they mean, how they are measured, and how the analysis will work. Share it with all members of the research team. Use the code book when entering your data. Tell the rest of us if you make changes to the code book or database. These simple tasks and communicative efforts can mean the difference between your statistics being run in five minutes versus five weeks.  SERIOUSLY.

    4. Write-up of findings for publication - We will give you an annotated write-up of your findings with statistical outputs and give you basic and unbiased interpretations of the statistical results of your study. We can help you write up the statistical methods and results sections of your abstracts and manuscripts. We can even help you design tables and graphs that will make your study findings more aesthetically and visually appealing to your audience.

    When it comes to authorship, if you feel that your statistical professional's contribution to the design, execution, and interpretation of your study warrants authorship, offer it to them. They will greatly appreciate it! However, YOU SHOULD NEVER BE REQUIRED TO GIVE US AUTHORSHIP JUST BECAUSE WE RAN YOUR STATISTICS FOR YOU.  IT IS UNETHICAL FOR US TO REQUIRE AUTHORSHIP FOR DOING OUR JOB. THAT IS, IF OUR JOB IS TO RUN STATISTICS IN YOUR EMPIRICAL OR CLINICAL ENVIRONMENT.          
  • Published on

    Effect size, sample size, and statistical power

    Effect Size Measurement Research Design Sample Size Statistical Power Analysis

    Effect size, sample size, and statistical power

    Choose an effect size to maximize statistical power and decrease sample size

    Effect size, sample size, and statistical power are nebulous empirical constructs that require a strong working knowledge of each in a conceptual fashion.  Also, there are basic interdependent relationships that exist amongst the three constructs. A change in one will ALWAYS exact a predictable and static change in the other two.

    An effect size is the hypothesized difference expected by researchers in an a priori fashion between independent groups (between-subjects analysis), across time or observations (within-subjects analysis), or the magnitude and direction of association between constructs (correlations and multivariate analyses).

    Effect size planning is perhaps the HARDEST part of designing a research study. Oftentimes, researchers have NO IDEA of what type of effect size they are trying to detect.

    First and foremost, when researchers cannot state the hypothesized differences in their outcomes, an evidence-based measure of effect yielded from a published study that is theoretically or conceptually similar to the phenomenon of interest should be used. Using an evidence-based measure of effect in an a priori power analysis shows more empirical rigor on the part of the researchers and increases the internal validity of the study with the use of published values.

    Sample size is the absolute number of participants that are sampled from a given population for purposes of running inferential statistics. The nomenclature of the word, inferential, denotes the basic empirical reasoning that we are drawing a representative sample from a population and then conducting statistics in order to make inferences back to said population. An important part of preliminary study planning is to specify the inclusion and exclusion criteria for participation in your study and then getting an idea of how large a participant pool you have available to you from which to draw a sample for purposes of running inferential statistics.

    Due to the underlying algebra associated with mathematical science, large sample sizes will drastically increase your chances of detecting a statistically significant finding, or in other terms, drastically increase your statistical power. Large sample sizes will also allow you to detect both large and small effect sizes, regardless of scale of measurement of the outcome, research design, and/or magnitude, variance, and direction of the effect. Small sample sizes will decrease your chances of detecting statistically significant differences (statistical power), especially with categorical and ordinal outcomes, between-subjects and multivariate designs, and small effect sizes.

    Statistical power is the chance you have as a researcher to reject the null hypothesis, given that the treatment effect actually exists in the population. Basically, statistical power is the chance you have of finding a significant difference or main effect when running statistical analyses.  Statistical power is what you are interested in when you ask, "How many people do I need to find significance?"

    In the applied empirical sense, measuring for large effect sizes increases statistical power. Trying to detect small effect sizes will decrease your statistical power. Continuous outcomes increase statistical power because of increased precision and accuracy in measurement. Categorical and ordinal outcomes decrease statistical power because of decreased variance and objectivity of measurement. Within-subjects designs generate more statistical power due to participants serving as their own controls. Between-subjects and multivariate designs require more observations to detect differences and therefore decrease statistical power.