Sample size for Wilcoxon test
Effect size is the difference in means or medians between two observations of the outcome
In order to run a sample size calculation for a Wilcoxon test, researchers will need to seek out evidence that provides the means and standard deviations of the outcome at two different observations. The absolute difference between the two mean values and their respective variances is an evidence-based measure of effect. Researchers should seek out articles that are conceptually or theoretically similar to the study of interest and use the values reported in the articles to calculate sample size for a Wilcoxon test.
Most ordinal outcomes are reported as medians and interquartile ranges, and those are the correct statistics for purposes of reporting in a publication. However, when it comes to sample size calculations, researchers may be required to make an informed hypothesis based on the means and standard deviations that are reported in a published article. A good rule of thumb is to overestimate the variance of the effect size. Researchers do this because it forces them to collect more observations of the outcome, which in turn leads to more precise and accurate measures of effect.
For example, let's say that researchers find quality evidence that people in the treatment group have an average pain score of 7.1 with a standard deviation of 1.6 at baseline and people in the control group have an average pain score of 4.3 with a standard deviation of 1.1. There is an evidenced-based measure of effect of 2.8 (7.1 - 4.3 = 2.8) in this instance. Researchers could enter these values into G*Power and know exactly how many observations of the outcome They would need to detect the 2.8 treatment effect.
Most ordinal outcomes are reported as medians and interquartile ranges, and those are the correct statistics for purposes of reporting in a publication. However, when it comes to sample size calculations, researchers may be required to make an informed hypothesis based on the means and standard deviations that are reported in a published article. A good rule of thumb is to overestimate the variance of the effect size. Researchers do this because it forces them to collect more observations of the outcome, which in turn leads to more precise and accurate measures of effect.
For example, let's say that researchers find quality evidence that people in the treatment group have an average pain score of 7.1 with a standard deviation of 1.6 at baseline and people in the control group have an average pain score of 4.3 with a standard deviation of 1.1. There is an evidenced-based measure of effect of 2.8 (7.1 - 4.3 = 2.8) in this instance. Researchers could enter these values into G*Power and know exactly how many observations of the outcome They would need to detect the 2.8 treatment effect.
The steps for calculating sample size for a Wilcoxon test in G*Power
1. Start up G*Power.
2. Under the Test family drop-down menu, select t tests.
3. Under the Statistical test drop-down menu, select Means: Wilcoxon signed-rank test (matched pairs).
4. Under the Type of power analysis drop-down menu, select A priori: Compute required sample size - given alpha, power, and effect size.
5. If there is a directional hypothesis, under the Tail(s) drop-down menu, select One.
6. If there is a non-directional hypothesis, under the Tail(s) drop-down menu, select Two.
7. Under the Parent distribution drop-down menu, select Normal, unless researchers want to change the distribution according to the current empirical or clinical context.
8. Click the Determine button.
9. Enter the mean for the first observation into the Mean group 1 box. Example: "7.1"
10. Enter the mean for the second observation into the Mean group 2 box. Example: "4.3"
11. Enter the standard deviation associated with the mean of the first observation into the SD insert group 1 box. Example: "1.6"
12. Enter the standard deviation associated with the mean of the second observation into the SD insert group 2 box. Example: "1.1"
13. Click Calculate.
14. Click Calculate and transfer to main window.
15. Leave the alpha value at 0.05, unless researchers want to change the alpha value according to the current empirical or clinical context.
16. Enter ".80" into the Power (1-beta err prob) box, unless researchers want to change the power according to the current empirical or clinical context.
17. Click Calculate.
2. Under the Test family drop-down menu, select t tests.
3. Under the Statistical test drop-down menu, select Means: Wilcoxon signed-rank test (matched pairs).
4. Under the Type of power analysis drop-down menu, select A priori: Compute required sample size - given alpha, power, and effect size.
5. If there is a directional hypothesis, under the Tail(s) drop-down menu, select One.
6. If there is a non-directional hypothesis, under the Tail(s) drop-down menu, select Two.
7. Under the Parent distribution drop-down menu, select Normal, unless researchers want to change the distribution according to the current empirical or clinical context.
8. Click the Determine button.
9. Enter the mean for the first observation into the Mean group 1 box. Example: "7.1"
10. Enter the mean for the second observation into the Mean group 2 box. Example: "4.3"
11. Enter the standard deviation associated with the mean of the first observation into the SD insert group 1 box. Example: "1.6"
12. Enter the standard deviation associated with the mean of the second observation into the SD insert group 2 box. Example: "1.1"
13. Click Calculate.
14. Click Calculate and transfer to main window.
15. Leave the alpha value at 0.05, unless researchers want to change the alpha value according to the current empirical or clinical context.
16. Enter ".80" into the Power (1-beta err prob) box, unless researchers want to change the power according to the current empirical or clinical context.
17. Click Calculate.
Based on the values above in the example and the 17 steps above, a two-tailed 2.8 point effect with an alpha of .05 and a power of .80 yields a needed sample size of only 5 to detect that effect.
Click on the Statistics button to continue.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics