nm7 IMP method question

Dear users, I recently started using IMP methods offered by nm7. In the first IMP step, I had 100 iterations and the OBJ was quite stable already around 20 iterations. In the second IMP step to obtain SE of the estimate, as "$EST METHOD=IMP EONLY=1 INTERACTION NOABORT NITER=20 SIGL=4 NSIG=1 PRINT=1 ISAMPLE=300 DF=4 IACCEPT=1", the OBJ value kept increasing as iteration moved. at the end, the SE for the model parameter estimate was not bad (<50%RES) could someone help me understand why OBJ continue increasing in the second IMP step? thanks.

Dear Ethan. if you use the IMP method to estimate your parameters, you do not have to run a second IMP step. The ESTEP is run for every iteration and if the OFV is stable during the last iterations you can just use that. However the behavior you are describing is strange. It could be related to different settings for both steps. Can you please send the code for both $EST lines? Best regards, Sebastian Sebastian Ueckert, MSc, PhD student ----------------------------------------------- Pharmacometrics Research Group, Department of Pharmaceutical Biosciences, Uppsala University ----------------------------------------------- P.O. Box 591 SE-751 24 Uppsala Sweden ----------------------------------------------- [email protected] ----------------------------------------------- Work: +46-(0)18-471 4437

I've seen on a number of examples (of non-linear differential equations models) the IMP method diverge (by increasing the OF indefinitely, until it hits infinity). In those cases, IMPMAP was more successful. Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Sebastian Ueckert wrote: > Dear Wu, > > it is important to realize that the new estimation methods do not directly maximize the likelihood. The EM algorithm updates the population parameters by averaging the conditional mean/variances of the etas. Therefore, it is not guaranteed that the likelihood increases (OFV decreases). The classical method, opposed to that, are gradient based and only change the population parameters when the OFV decreases. > > There are several things you could try if the OFV is continuously increasing when using the IMP method: - make sure that your model is valid for unusual eta samples (prevent e.g. division by zero, logarithm of values smaller than zero, etc.) > > - change the initial values > - use a different estimation methods prior to IMP (e.g. FOCE, ITS) > - increase the number of individual samples (ISAMPLE) > - use a different sampling distribution (e.g. DF=1) > - try using the IMPMAP method instead > > It would be interesting to know if other users have additional suggestions. > > Best regards, > Sebastian > > Sebastian Ueckert, MSc, PhD student > ----------------------------------------------- > Pharmacometrics Research Group, > Department of Pharmaceutical Biosciences, > Uppsala University > ----------------------------------------------- > P.O. Box 591 > SE-751 24 Uppsala > Sweden > ----------------------------------------------- > [email protected] <mailto:[email protected]> > ----------------------------------------------- > Work: +46-(0)18-471 4437 >

Hi, SIGL=4 NSIG=1 seems low... Are you sure this is not simply due to the propagation of numerical errors caused by having to few significant digits? K. Regards, Xavier Leonid Gibiansky wrote: > I've seen on a number of examples (of non-linear differential equations models) the IMP method diverge (by increasing the OF indefinitely, until it hits infinity). In those cases, IMPMAP was more successful. > > Leonid > > -------------------------------------- > Leonid Gibiansky, Ph.D. > President, QuantPharm LLC > web: www.quantpharm.com > e-mail: LGibiansky at quantpharm.com > tel: (301) 767 5566 > > Sebastian Ueckert wrote: > > > Dear Wu, > > > > it is important to realize that the new estimation methods do not directly maximize the likelihood. The EM algorithm updates the population parameters by averaging the conditional mean/variances of the etas. Therefore, it is not guaranteed that the likelihood increases (OFV decreases). The classical method, opposed to that, are gradient based and only change the population parameters when the OFV decreases. > > > > There are several things you could try if the OFV is continuously increasing when using the IMP method: - make sure that your model is valid for unusual eta samples (prevent e.g. division by zero, logarithm of values smaller than zero, etc.) > > > > - change the initial values > > - use a different estimation methods prior to IMP (e.g. FOCE, ITS) > > - increase the number of individual samples (ISAMPLE) > > - use a different sampling distribution (e.g. DF=1) > > - try using the IMPMAP method instead > > > > It would be interesting to know if other users have additional suggestions. > > > > Best regards, > > Sebastian > > > > Sebastian Ueckert, MSc, PhD student > > ----------------------------------------------- > > Pharmacometrics Research Group, > > Department of Pharmaceutical Biosciences, > > Uppsala University > > ----------------------------------------------- > > P.O. Box 591 > > SE-751 24 Uppsala > > Sweden > > ----------------------------------------------- > > [email protected] <mailto:[email protected]> > > ----------------------------------------------- > > Work: +46-(0)18-471 4437 > > > > On Wed, May 12, 2010 at 5:30 PM, Xiao Hu < [email protected] < mailto: [email protected] >> wrote: > > > > *Dear Sebastian and Ethan,* > > > > I have encountered similar situation in the same estimation step. It also occurred using Baysian method. The MCMCOBJ just kept > > > > increasing from approximately -100 to 2000, while CONDITIONAL LAP > > and FOCE converged successfully. I found different methods work for > > different dataset and model structures. Changing initial estimation > > did not help much. Any insight on why OBJ and MCMCOBJ keep > > > > increasing, and how to prevent it would be helpful. Thanks in advance. > > > > Best regards, > > Shelley > > > > ======================================== > > Xiao Hu (Shelley), Ph.D. > > Scientist, > > Development Pharmacokinetics & Disposition > > Biogen Idec, Inc. > > 14 Cambridge Center > > Cambridge, MA 02142 > > > > *Sebastian Ueckert <[email protected] > > <mailto:[email protected]>>* > > Sent by: [email protected] > > <mailto:[email protected]> > > > > 12-May-2010 11:08 AM > > > > Message Size: *6.2 KB* > > > > To > > > > nmusers <[email protected] <mailto:[email protected]>> > > cc > > > > Subject > > > > Re: [NMusers] nm7 IMP method question > > > > Dear Ethan. > > if you use the IMP method to estimate your parameters, you do not > > have to run a second IMP step. The ESTEP is run for every iteration > > and if the OFV is stable during the last iterations you can just use > > that. However the behavior you are describing is strange. It could > > be related to different settings for both steps. Can you please send > > the code for both $EST lines? > > > > Best regards, > > Sebastian > > > > Sebastian Ueckert, MSc, PhD student > > ----------------------------------------------- > > Pharmacometrics Research Group, > > Department of Pharmaceutical Biosciences, > > Uppsala University > > ----------------------------------------------- > > P.O. Box 591 > > SE-751 24 Uppsala > > Sweden > > -----------------------------------------------_ > > [email protected]_ > > <mailto:[email protected]> > > ----------------------------------------------- > > Work: +46-(0)18-471 4437 > > > > On Wed, May 12, 2010 at 4:13 PM, Ethan Wu <[email protected]_ > > <mailto:[email protected]>> wrote: > > IACCEPT

Desr Ethan, It is my understanding that you should do an IMP step with EONLY=1 in order to get an OFV to be used for hypothesis testing after a SAEM step, as the SAEM OFV cannot be used for likelihood ratio tests. I think your problem is that the first IMP step has converged (you asked for CTYPE=3), but in the second IMP step you are not specifying any convergence criteria causing your OFV to wander off (excuse the technical language). Best wishes, Joe Joseph F Standing Department of Pharmacy Great Ormond Street Hospital for Children London WC1N 3JH

Desr Ethan, It is my understanding that you should do an IMP step with EONLY=1 in order to get an OFV to be used for hypothesis testing after a SAEM step, as the SAEM OFV cannot be used for likelihood ratio tests. I think your problem is that the first IMP step has converged (you asked for CTYPE=3), but in the second IMP step you are not specifying any convergence criteria causing your OFV to wander off (excuse the technical language). Best wishes, Joe Joseph F Standing Department of Pharmacy Great Ormond Street Hospital for Children London WC1N 3JH

Ethan: We have noticed this occurs with IMP on a few other problems. We have modified the code so this occurs less often, and the change will be in NONMEM 7.2, scheduled for January. Meanwhile, as Sebastian suggested, only SAEM needs an IMP EONLY=1 step. The importance sampler itself produces its own standard errors, so your first IMP statement with EONLY=0 (implied) should be enough. Robert J. Bauer, Ph.D. Vice President, Pharmacometrics ICON Development Solutions Tel: (215) 616-6428 Mob: (925) 286-0769 Email: Robert.Bauer Web: www.icondevsolutions.com

Ethan: We have noticed this occurs with IMP on a few other problems. We have modified the code so this occurs less often, and the change will be in NONMEM 7.2, scheduled for January. Meanwhile, as Sebastian suggested, only SAEM needs an IMP EONLY=1 step. The importance sampler itself produces its own standard errors, so your first IMP statement with EONLY=0 (implied) should be enough. Robert J. Bauer, Ph.D. Vice President, Pharmacometrics ICON Development Solutions Tel: (215) 616-6428 Mob: (925) 286-0769 Email: [email protected] Web: www.icondevsolutions.com