Split-group validation
Compare independent groups of randomly assigned participants
The split-group validation method is a fairly straightforward process. Run inferential statistics using your dataset. Then, randomly assign members of the dataset to either a derivation group or a confirmatory group. The split between the two independent groups can be 50/50, 60/40, or 70/30. Run the same inferential statistics on both the derivation group and the confirmatory group. If the findings are similar, then you can assume validity for the original findings. That is the basic methodology for split-group validation. Use randomizer.org as a platform for randomization in split-group validation.
Split-group validation in SPSS
Unlike the Bootstrap... technique available in SPSS, the split-group validation technique is not a button that is clicked. Instead, it is a simple recode conducted using the Data drop-down menu at the top of the screen.
Here are the methods for splitting the study participants into the derivation and confirmatory datasets:
1. Click on the Data drop-down menu.
2. Click on Select Cases... to select it.
3. In the Select table, click on the Random sample of cases button to select it.
4. Click on the Sample... button.
5. In the Sample Size table, click on the Approximately button to select it.
6. If researchers want to use a 50/50 split-group method, type the number 50 into the % of all cases box. For 60/40, type the number 60. For 70/30, type the number 70.
7. Click Continue.
8. Click OK.
9. Back in the Data View window, scroll all the way to the right of the database to find a variable denoted as "filter_$"
10. The observations denoted as "0" can be codified as the derivation group and the observations denoted as "1" can be codified as the confirmatory group.
11. Run statistical analyses on the two groups independently. If the results are similar, then validity evidence for the original statistical findings is assumed.
Here are the methods for splitting the study participants into the derivation and confirmatory datasets:
1. Click on the Data drop-down menu.
2. Click on Select Cases... to select it.
3. In the Select table, click on the Random sample of cases button to select it.
4. Click on the Sample... button.
5. In the Sample Size table, click on the Approximately button to select it.
6. If researchers want to use a 50/50 split-group method, type the number 50 into the % of all cases box. For 60/40, type the number 60. For 70/30, type the number 70.
7. Click Continue.
8. Click OK.
9. Back in the Data View window, scroll all the way to the right of the database to find a variable denoted as "filter_$"
10. The observations denoted as "0" can be codified as the derivation group and the observations denoted as "1" can be codified as the confirmatory group.
11. Run statistical analyses on the two groups independently. If the results are similar, then validity evidence for the original statistical findings is assumed.
Hire A Statistician
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D., PStat will provide the following statistical consulting services for undergraduate and graduate students at $100/hour. Secure checkout is available with Stripe, Venmo, Zelle, or PayPal.
- Statistical Analysis
- Research Design
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Survey Design and Psychometrics