Re: FW: PPC

Mohamed, When the number of subjects is small then any confidence interval is going to be wide and probably no-one is really interested in it. With studies more suitable for population analysis (at least 25 subjects and preferably over 100 if you want to look for covariate effects) then the CIs may be more interesting. With linear models or parameters which are nearly linear in non-linear models then I would expect quite good agreement between CIs obtained by bootstrapping and by using NONMEM SEs. But the models get interesting when one tries to estimate non-linear parameters e.g. EC50 in an Emax model. In that case the CIs will often be asymmetrical and the normal distribution assumption used to compute SEs from CIs will be wrong. Leonid does not discuss the issue of assymetry of CIs in his poster -- but when I look at Figure 3 I see evidence for disagreement between bootstrap and NONMEM SE based CIs. The scatter of bootstrap points relative to the solid line shows an excess of bootstrap upper CI values above the SE prediction. For the lower CI prediction there also seem to be more bootstrap values above the SE predictions. Its hard to be sure that these upper and lower bootstrap predictions belong to the same parameters but if so this would be evidence for asymmetry of the bootstrap CI. This is exactly what one would expect because the SE method has to assume symmetrical CIs yet the bootstrap estimate is not restricted in this way. I think this poster is a nice example of why correlation coefficients are a very poor way to compare predictions (as pointed out by Sheiner and Beal in their classic paper Sheiner LB, Beal SL. Some Suggestions for Measuring Predictive Performance. J Pharmacokinet Biopharm. 1981;9(4):503-12.). A better way would be to compute the prediction error for the absolute larger CI arm and smaller CI arm obtained by bootstrapping to the symmetrial CI from the SE. If bootstraps CIs are indeed asymmetrical then there would be a difference shown by the mean prediction error ('bias'). Note that I use absolute value larger and smaller CI arm to refer to the larger or smaller part of the CI that is constructed around zero. I dont mean the upper and lower parts of the CI interval. Best wishes, Nick Leonid Gibiansky wrote: > I cannot make any general statements but here is the summary of the 13 different models that I tested for comparison of bootstrap and nonmem CI. > > http://www.quantpharm.com/pdf_files/2572-GibianskyPage2007Poster2007final.pdf > > Note that all bootstrap samples were appropriately stratified by major covariates (such as study, dose, weight as necessary, etc.). > > Leonid > > -------------------------------------- > Leonid Gibiansky, Ph.D. > President, QuantPharm LLC > web: www.quantpharm.com > e-mail: LGibiansky at quantpharm.com > tel: (301) 767 5566 > > [EMAIL PROTECTED] wrote: > > > Dear Dr. Holford, > > > > Please correct me if I am wrong, however my understanding is that asymptotic distribution implied by NONMEM's covariance step approaches normality as the sample size gets larger or we have more data. However, a non parametric bootstrap distribution may have poor coverage with a small sample size as well, since it relies on sampling subjects with repalcement in the data set. So both distributions have problems when sample size is small (e.g. N<30). Therefore I would think when N is large the wald based Confidence Intervals from NONMEM are appropriate enough. It would be helpful to know the criteria when generating a non parametric bootstrap distribution is really advantageous. > > > > Thanks, Mohamed > > Quoting Nick Holford <[EMAIL PROTECTED]>: -- Nick Holford, Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [EMAIL PROTECTED] tel:+64(9)373-7599x86730 fax:+64(9)373-7090 http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford