Sample size for Mann-Whitney U
Effect size is the difference in means or medians between two groups on the outcome
In order to run an a priori sample size calculation for a Mann-Whitney U, researchers will need to seek out evidence that provides the means and standard deviations of the outcome in the treatment group and the control group. The absolute difference between the two mean values and their respective variances dictates measure of effect. Find an article in the literature that is similar to the study being conducted and use the reported medians and interquartile ranges to run a sample size calculation for Mann-Whitney U.
Most ordinal outcomes are reported as medians and interquartile ranges, and those are the correct statistics for purposes of reporting in a publication. However, when it comes to sample size calculations for Mann-Whitney U, researchers may be required to make an informed hypothesis based on means and standard deviations that are reported. A good rule of thumb is to overestimate the variance of the effect size. Researchers do this because it forces them to collect more observations of the outcome, which in turn leads to more precise and accurate measures of effect.
For example, let's say that researchers find quality evidence that people in the treatment group have an average pain score of 2.3 with a standard deviation of 1.2 and people in the control group have an average pain score of 4.9 with a standard deviation of 1.8). They have an evidenced-based measure of effect of 2.6 (4.9 - 2.3 = 2.6). They could enter these values into G*Power and know exactly how many observations of the outcome would be needed to detect that 2.6 treatment effect.
Most ordinal outcomes are reported as medians and interquartile ranges, and those are the correct statistics for purposes of reporting in a publication. However, when it comes to sample size calculations for Mann-Whitney U, researchers may be required to make an informed hypothesis based on means and standard deviations that are reported. A good rule of thumb is to overestimate the variance of the effect size. Researchers do this because it forces them to collect more observations of the outcome, which in turn leads to more precise and accurate measures of effect.
For example, let's say that researchers find quality evidence that people in the treatment group have an average pain score of 2.3 with a standard deviation of 1.2 and people in the control group have an average pain score of 4.9 with a standard deviation of 1.8). They have an evidenced-based measure of effect of 2.6 (4.9 - 2.3 = 2.6). They could enter these values into G*Power and know exactly how many observations of the outcome would be needed to detect that 2.6 treatment effect.
The steps for calculating sample size for a Mann-Whitney U in G*Power
1. Start up G*Power.
2. Under the Test family drop-down menu, select t tests.
3. Under the Statistical test drop-down menu, select Means: Wilcoxon-Mann-Whitney test (two groups).
4. Under the Type of power analysis drop-down menu, select A priori: Compute required sample size - given alpha, power, and effect size.
5. If there is a directional hypothesis, under the Tail(s) drop-down menu, select One.
6. If there is a non-directional hypothesis, under the Tail(s) drop-down menu, select Two.
7. Under the Parent distribution drop down menu, select Normal, unless researchers want to change the distribution according to the current empirical or clinical context.
8. Click the Determine button.
9. Enter the mean for the treatment group into the Mean group 1 box. Example: "2.3"
10. Enter the mean for the control group into the Mean group 2 box. Example: "4.9"
11. Enter the standard deviation associated with the mean of the treatment group into the SD insert group 1 box. Example: "1.2"
12. Enter the standard deviation associated with the mean of the control group into the SD insert group2 box. Example: "1.8"
13. Click Calculate.
14. Click Calculate and transfer to main window.
15. Leave the alpha value at 0.05, unless researchers want to change the alpha value according to the current empirical or clinical context.
16. Enter ".80" into the Power (1-beta err prob) box, unless researchers want to change the power according to the current empirical or clinical context.
17. If researchers have exactly equally sized groups, then leave the Allocation ratio N2/N1 value at "1." If researchers have unequally sized groups, then divide the sample size of the treatment group by the sample size of the control group and enter that value into the box.
18. Click Calculate.
2. Under the Test family drop-down menu, select t tests.
3. Under the Statistical test drop-down menu, select Means: Wilcoxon-Mann-Whitney test (two groups).
4. Under the Type of power analysis drop-down menu, select A priori: Compute required sample size - given alpha, power, and effect size.
5. If there is a directional hypothesis, under the Tail(s) drop-down menu, select One.
6. If there is a non-directional hypothesis, under the Tail(s) drop-down menu, select Two.
7. Under the Parent distribution drop down menu, select Normal, unless researchers want to change the distribution according to the current empirical or clinical context.
8. Click the Determine button.
9. Enter the mean for the treatment group into the Mean group 1 box. Example: "2.3"
10. Enter the mean for the control group into the Mean group 2 box. Example: "4.9"
11. Enter the standard deviation associated with the mean of the treatment group into the SD insert group 1 box. Example: "1.2"
12. Enter the standard deviation associated with the mean of the control group into the SD insert group2 box. Example: "1.8"
13. Click Calculate.
14. Click Calculate and transfer to main window.
15. Leave the alpha value at 0.05, unless researchers want to change the alpha value according to the current empirical or clinical context.
16. Enter ".80" into the Power (1-beta err prob) box, unless researchers want to change the power according to the current empirical or clinical context.
17. If researchers have exactly equally sized groups, then leave the Allocation ratio N2/N1 value at "1." If researchers have unequally sized groups, then divide the sample size of the treatment group by the sample size of the control group and enter that value into the box.
18. Click Calculate.
Based on the example and steps presented above, with a two-tailed test based in a normal distribution with those means and standard deviations, an alpha of .05 and power of .80 and equally sized groups, researchers would need a total of 14 participants in the study with seven in each group.
Click on the Statistics button to continue.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics
