RE: Bootstrap resampling!

From: "KOWALSKI, KENNETH G. [PHR/1825]" <kenneth.g.kowalski@pharmacia.com> Subject: RE: Bootstrap resampling! Date: Thu, 29 Mar 2001 08:42:39 -0600 Yes, what Leonid is describing is a form of randomization test. It is a way to get a post-hoc assessment of the alpha level (false positive rate) of the test. If one completely disassociates the covariate from the patient then for the bootstrap data sets testing say the effect of gender, we should get a false positive rate of about 5% if we are using a standard chi-square critical value of 3.84. However, in the presence of model misspecification the actual false positive rate can be much higher. I have a simulation example where the false positive rate can be as high as 25% when using a chi-square test presumably controlled at the 5% significance level when there is known model misspecification (see Kowalski and Hutmacher, Stats. in Med, 2001;20:75-91). By disassociating the covariates from the patients in doing the bootstrapping one can essentially characterize the empirical distribution of the test statistc under the null hypothesis (e.g., no gender difference) and assess whether the distribution of bootstrap test statistics really follow the chi-square distribution. Ken