RE: Permutation test with small number of possible permutations

From: "Mouksassi Mohamad-Samer" mohamad-samer.mouksassi@umontreal.ca Subject: RE: [NMusers] Permutation test with small number of possible permutations Date: Fri, November 26, 2004 11:05 pm Leonid, First thanks for the script. Is there a general formula for the number of permutations? If class A is assigned to 5678 or 1234 it would be different because we would have then : A A A A B B B B B B B B A A A A these are not equivalent, am I wrong? ID 1 2 3 4 5 6 7 8 The p value should be the same but the variance of it would be much bigger as it is (p*(1-p))/n for a computed 0.05 sd(p)= 0.00689 (1000) sd(p)= 0.036 (35) Too large ....to be useful sd(p)= 0.026 (70) Nick, maybe I used the wrong word criticized, Stu Beal said that this may not hold in all the settings and he added it is instructive to consider another idea for calibrating the critical value as the posterior predictive check. Samer