Statistical power and sample size
Within the current empirical or clinical context, will there be a relatively small or large sample?
Statistics is a flawed mathematical science where certain assumptions have to be met in order to make inferences back to a population. There are also errors in measurement, selection biases, and observation biases associated with conducting research which can lead to erroneous statistical findings.
There is one axiom that holds true in statistical analysis of research data. With increased sample sizes, statistical power will increase. With hundreds or thousands of observations, significant effects are just easier to detect. These large sample sizes also provide the most precise measure of treatment effect in regards to 95% confidence intervals. This will even be true when making empirical decisions that normally decrease statistical power like 1) using categorical and ordinal outcomes, 2) choosing between-subjects and multivariate designs, 3) measuring for small effect sizes, and 4) hypothesizing extensive variance in the outcome.
When an outcome is rare, or when fewer than 20-30 observations of an outcome can be collected in between-subjects or multivariate research designs, statistical power will decrease and the ability to detect significant treatments effects will be greatly decreased. Small sample sizes often lead to Type II errors, or failing to reject the null hypothesis when you should (false negative). Post hoc power analyses should be conducted when small sample sizes are used in research. This gives context to non-significant statistical findings.
There is one axiom that holds true in statistical analysis of research data. With increased sample sizes, statistical power will increase. With hundreds or thousands of observations, significant effects are just easier to detect. These large sample sizes also provide the most precise measure of treatment effect in regards to 95% confidence intervals. This will even be true when making empirical decisions that normally decrease statistical power like 1) using categorical and ordinal outcomes, 2) choosing between-subjects and multivariate designs, 3) measuring for small effect sizes, and 4) hypothesizing extensive variance in the outcome.
When an outcome is rare, or when fewer than 20-30 observations of an outcome can be collected in between-subjects or multivariate research designs, statistical power will decrease and the ability to detect significant treatments effects will be greatly decreased. Small sample sizes often lead to Type II errors, or failing to reject the null hypothesis when you should (false negative). Post hoc power analyses should be conducted when small sample sizes are used in research. This gives context to non-significant statistical findings.
Sample sizes
Small sample sizes are defined as less than 50 independent observations in a between-subjects, mixed, or multivariate design. In within-subjects designs, a small sample size could be defined as having less than 20 observations.
A relatively large sample size or pool of participants could defined as more than 50 independent observations in a between-subjects design, more than 100 observations in a mixed or multivariate design. In within-subjects designs, a large sample size could be defined as having more than 20 observations.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics