The impact of distributional shape on the power of randomization tests for two independent groups: a simulation study using small balanced samples

Abstract

The importance of randomization tests is very well known in experimental research, particularly in biometry. The aim of the resent research is to evaluate the impact of distributional shape on the power of the randomization test for difference between the means of two independent groups (with n1=n2=16). To manipulate shape in terms of asymmetry and kurtosis, we used g-and-h distributions. We evaluated the power of the randomization test, and also the power of the Student-t test, as a comparison standard, with data simulated from 12 g-and-h distributions for seven values of effect size. For each condition, we generated 20000 samples, and for each one the power of randomization tests was estimated using 1000 permutations. We set the value of Type I error probability at 0.05. The results show gains in power for both tests with increasing skewness and/or kurtosis, with a slight advantage for the randomization tests over the Student-t test.