Minimization Terminated in Bootstrapping

From: [email protected] [mailto:[email protected]] On Behalf Of siwei Dai Sent: Tuesday, October 28, 2014 9:00 AM To: [email protected] Subject: [NMusers] Minimization Terminated in Bootstrapping Dear NM users: I have a data set where the absorption phase is highly variable. A complicated absorption model (the TRANSIT model) significantly improve the model fit (OFV drops over 250) compared to a 1st-order absorption model with lag time. The complicated final model converged and the covariance step was successful. The condition number is low (less than 10), although there were many runs with rounding errors during model development. However, in the bootstrap analysis,about 600 out of 1000 iterations had minimization terminated which leads to a successful rate of only 40%. My question is: is there any consensus regarding the acceptable successful rate for bootstrap analysis? I understand 40% is very low. Should I reject this model then? Any suggestions will be greatly appreciated. Thanks, Siwei

On Wed, Oct 29, 2014 at 11:58 AM, Bonate, Peter <[email protected]> wrote: > Not just yet. I’ve seen many people just naively bootstrap their > dataset. It may be that your bootstrap datasets do not adequately > represent model development dataset. For example suppose you had 75% > sparse data and 25% rich data in your model development set. It is > imperative that your bootstrap dataset maintain this ratio so you need to > stratify by data richness. Otherwise you could get a bootstrap dataset that > is predominantly sparse data which would not support such a complicated > absorption model. The same holds true if there are important covariates in > your model. > > > > pete > > > > > > Peter L. Bonate, PhD > > Senior Director > > Global Head - Pharmacokinetics, Modeling, and Simulation > > Global Clinical Pharmacology & Exploratory Development > > > > Astellas > > 1 Astellas Way, 2N.184 > > Northbrook, Il 60062 > > phone: 224-205-5855 > > fax: 224-205-5914 > > email: [email protected] > > > > > > *From:* [email protected] [mailto:[email protected]] > *On Behalf Of *siwei Dai > *Sent:* Tuesday, October 28, 2014 9:00 AM > *To:* [email protected] > *Subject:* [NMusers] Minimization Terminated in Bootstrapping > > > > Dear NM users: > > > > I have a data set where the absorption phase is highly variable. A > complicated absorption model (the TRANSIT model) significantly improve the > model fit (OFV drops over 250) compared to a 1st-order absorption model > with lag time. The complicated final model converged and the covariance > step was successful. The condition number is low (less than 10), although > there were many runs with rounding errors during model development. > > > > However, in the bootstrap analysis,about 600 out of 1000 iterations had > minimization terminated which leads to a successful rate of only 40%. > > > > My question is: is there any consensus regarding the acceptable successful > rate for bootstrap analysis? I understand 40% is very low. Should I reject > this model then? > > > > Any suggestions will be greatly appreciated. > > > Thanks, > > > > Siwei >

On 30 October 2014 21:17, siwei Dai <[email protected]> wrote: > Hi, Dr. Bonate: > > Thank you very much for your comments. There are some more sparse data in > the data set and I wasn't aware that I should do stratification for > bootstrapping, so thank you for pointing it out. > > Best regards, > > Siwei > > On Wed, Oct 29, 2014 at 11:58 AM, Bonate, Peter <[email protected] > > wrote: > >> Not just yet. I’ve seen many people just naively bootstrap their >> dataset. It may be that your bootstrap datasets do not adequately >> represent model development dataset. For example suppose you had 75% >> sparse data and 25% rich data in your model development set. It is >> imperative that your bootstrap dataset maintain this ratio so you need to >> stratify by data richness. Otherwise you could get a bootstrap dataset that >> is predominantly sparse data which would not support such a complicated >> absorption model. The same holds true if there are important covariates in >> your model. >> >> >> >> pete >> >> >> >> >> >> Peter L. Bonate, PhD >> >> Senior Director >> >> Global Head - Pharmacokinetics, Modeling, and Simulation >> >> Global Clinical Pharmacology & Exploratory Development >> >> >> >> Astellas >> >> 1 Astellas Way, 2N.184 >> >> Northbrook, Il 60062 >> >> phone: 224-205-5855 >> >> fax: 224-205-5914 >> >> email: [email protected] >> >> >> >> >> >> *From:* [email protected] [mailto:[email protected]] >> *On Behalf Of *siwei Dai >> *Sent:* Tuesday, October 28, 2014 9:00 AM >> *To:* [email protected] >> *Subject:* [NMusers] Minimization Terminated in Bootstrapping >> >> >> >> Dear NM users: >> >> >> >> I have a data set where the absorption phase is highly variable. A >> complicated absorption model (the TRANSIT model) significantly improve the >> model fit (OFV drops over 250) compared to a 1st-order absorption model >> with lag time. The complicated final model converged and the covariance >> step was successful. The condition number is low (less than 10), although >> there were many runs with rounding errors during model development. >> >> >> >> However, in the bootstrap analysis,about 600 out of 1000 iterations had >> minimization terminated which leads to a successful rate of only 40%. >> >> >> >> My question is: is there any consensus regarding the acceptable >> successful rate for bootstrap analysis? I understand 40% is very low. >> Should I reject this model then? >> >> >> >> Any suggestions will be greatly appreciated. >> >> >> Thanks, >> >> >> >> Siwei >> > >