Re: Bootstrap resampling! -> Randomization test

From: Nick Holford <n.holford@auckland.ac.nz> Subject: Re: Bootstrap resampling! -> Randomization test Date: Fri, 30 Mar 2001 07:04:06 +1200 Leonid, You are describing the randomization test procedure. I don't know if it is strictly correct to call it a bootstrap method (Steve Duffull has promised to return Davison's book and I will try to check this when it comes back). Two links which explain the RT and put it in perspective are: http://garnet.acns.fsu.edu/~pkelly/resampling.html http://davidmlane.com/hyperstat/B143907.html The purpose of the RT procedure is to determine the 'true' probability that you would reject the null hypothesis (e.g. H0: gender does not explain the variability in CL) when the null hypothesis was in fact true. The relevance to NONMEM users is very strong. If you use the FOCE method there are several people (e.g. Mats Karlsson's group in Uppsala) who have found that the chi square distribution is pretty good for predicting a P value (none of this has been published as far as I knows but if anyone know of a publication please tell us). But if you use FO then you need a larger difference in the objective function (deltaOBJ) to correctly reject the null hypothesis. Anybody using NONMEM is doing this kind of hypothesis test all the time. If you use FO and are relying on a deltaOBJ of 3.84 for a one parameter difference in the models then you are almost certainly drawing conclusions with a P value > 0.05. A test case I did with a one compartment model indicated that the P value associated with deltaOBJ of 3.84 was 0.08 and that I should use a deltaOBJ of 4.8 to reject the null with alpha=0.05. I have recently (this week) implemented the randomization test (RT) as part of Wings for NONMEM ( http://www.geocities.com/wfn2k/). I am still doing some testing but hope to release it by the beginning of next week. It will let you run the RT with any NMTRAN control stream and data set with a command such as: nmrt wt theopd 1000 where nmrt is the RT command, wt is the covariate to be randomized, theopd is the runname (the NMTRAN control stream) and 1000 is the number of replications. In order to evaluate the P value you will need to run at least 1000 replications so this procedure is only practical for NONMEM runs that complete within a few minutes or less. Nick Nick Holford, Divn Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand email:n.holford@auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556 http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm