RE: RE: Successful minimization and covariance

From: Mark Sale Date: May 24, 2012 technical Source: mail-archive.com
I see another long discussion with strong feelings on both side. WRT Stu Beals comments, I had the opportunity to discuss this with him once. His point (which is to my knowledge the only argument for a successful covariance per se) is that a covariance matrix that is not positive definite may represent a "saddle point" in the objective function surface - with 0 2nd derivative in some dimensions, to machine precision. This was about 25 years ago. Since then, I understand that the more modern minimization routines that NONMEM now uses pretty much preclude a saddle point. This leaves the other argument, that it isn't the covariance step per se that is useful but the information gained (SEE, eigenvalues etc) that are useful. But, if we believe that the positive definite-ness of a solution is in question, then the SEE and eigenvalues may also be in question. My experience, and a little preliminary data suggest that more modern diagnostics are better indicators of model stability and overall "goodness". It may be time to move on. Mark Sale MD President, Next Level Solutions, LLC www.NextLevelSolns.com 919-846-9185 A carbon-neutral company See our real time solar energy production at: http://enlighten.enphaseenergy.com/public/systems/aSDz2458
Quoted reply history
-------- Original Message -------- Subject: Re: [NMusers] RE: Successful minimization and covariance From: Fisher Dennis < [email protected] > Date: Thu, May 24, 2012 11:53 am To: Holford Nick < [email protected] > Cc: [email protected] Nick In fairness, Stu Beal advocated strongly for the position taken by Gianluca. However, my own experience is that runs that fail to converge sometimes (often?) converge after minor changes to the initial estimates. Dennis Dennis Fisher MD P < (The "P Less Than" Company) Phone: 1-866-PLessThan (1-866-753-7784) Fax: 1-866-PLessThan (1-866-753-7784) www.PLessThan.com On May 24, 2012, at 7:51 AM, Nick Holford wrote: > Gianluca, > > What is your experimental evidence to allow you to conclude that failure of convergence is due to over-parameterization? (instead of other things like to large value of NSIG) > > Nick > > On 24/05/2012 4:34 p.m., Nucci, Gianluca wrote: >> >> I would not worry about the validity of your bootstrapped CI (and I would include all the runs) but I think you have to worry that your model is seriously over-parameterized if 60% of bootstrapped runs fail to converge. It does not mean it is a bad model – just that the data does not permit to fit all the parameters and you should consider fixing something, or use prior information or add data from more intensively sampled studies. >> Best Regards >> >> Gianluca >> >> Gianluca Nucci, PhD >> >> Clinical Pharmacology >> >> Pfizer PharmaTherapeutics R&D >> >> 620 Memorial Drive, >> >> Cambridge, MA 02139 >> >> Room # 464 >> >> Office 617-551-3525 >> >> Mobile 860-405-4824 >> >> Fax 860-686-8225 >> >> *From: * [email protected] [ mailto: [email protected] ] *On Behalf Of *Ayyappa Chaturvedula >> *Sent:* Thursday, May 24, 2012 8:33 AM >> *To:* [email protected] >> *Subject:* [NMusers] Successful minimization and covariance >> >> Dear Group, >> >> This is a topic that has been discussed and different schools of thinking exist to my knowledge. But, I want to restate my case and get some opinions. The question is about how important to have successful minimization and covariance if diagnostics make sense. I have developed a two compartment model with a Phase 3 trial data and minimization was successful but covariance step was not. I went ahead and did a 1000 run bootstrap and wanted to get the confidence intervals of parameters. There are 60% runs that are not successfully minimized and many other do not have covariance step successful. I put together CI from the runs that have successful minimization and also including all 1000 runs. There is no difference in the parameter estimate or the confidence interval (less than 5% change in numbers). The model diagnostics look good including VPC, NPDE plots, basic gof and a simulation to explain another trial data. Now, my question is in this particular case do I have to worry further to make the successful covariance step and increase the number of runs that gets successfully minimized in the bootstrap even though I cannot see much difference in the parameter estimates, diagnostics? My bottom line is not going to change in anyway. I appreciate your expert opinions. >> >> Regards, >> >> Ayyappa >> > > -- > Nick Holford, Professor Clinical Pharmacology > > First World Conference on Pharmacometrics, 5-7 September 2012 > Seoul, Korea http://www.go-wcop.org > > Dept Pharmacology& Clinical Pharmacology, Bldg 505 Room 202D > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 > email: [email protected] > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > > >
May 24, 2012 Ayyappa Chaturvedula Successful minimization and covariance
May 24, 2012 Nick Holford Re: Successful minimization and covariance
May 24, 2012 Gianluca Nucci RE: Successful minimization and covariance
May 24, 2012 Fisher Dennis Re: RE: Successful minimization and covariance
May 24, 2012 Mark Sale RE: RE: Successful minimization and covariance
May 24, 2012 Unknown Re: Successful minimization and covariance
May 24, 2012 Kenneth Kowalski RE: Successful minimization and covariance
May 24, 2012 Erik Olofsen RE: Successful minimization and covariance