NM7 Question on METHOD=IMP

Dear NM7 users, According to the examples in the users guide, the IMP step followed the SAEM step with EONLY=1 to evaluate the objective function. In all of my models (for the same data set), I have these two $EST steps: $EST METHOD=SAEM INTER FILE=saem.txt NBURN=3000 NITER=2000 PRINT=5 CTYPE=3 GRD=DDDDDDDDDSSDDD SEED=20091013 ISAMPLE=2 $EST METHOD=IMP EONLY=1 FILE=imp.txt NITER=300 ISAMPLE=3000 PRINT=1 In my base model, the OBJ trend in the IMP step is as follows (truncated here): iteration 296 OBJ= -1796.0722480934633 iteration 297 OBJ= -1802.1845380634943 iteration 298 OBJ= -1795.4946269639131 iteration 299 OBJ= -1795.8286754523187 Elapsed estimation time in seconds: 230513 iteration 300 OBJ= -1791.4880075805286 When I add IOV on one of the parameters (to the base model), I got the following trend, which showed the OBJ became worse: iteration 296 OBJ= 52215.178158178285 iteration 297 OBJ= 51758.149614882743 iteration 298 OBJ= 51850.531967877905 iteration 299 OBJ= 53377.558672094958 Elapsed estimation time in seconds: 314605 iteration 300 OBJ= 53030.267320528052 When I compared the GOF plots and parameter estimates for these two models, they looked pretty much the same. I could not find any obvious differences between them. How would I interpret/compare the OBJ for these 2 models? What could be the reasons for such difference in the OBJ for the 2 models? Thank you in advance for your help, -Thuy

Dear Thuy: The OBJ of the IOV model is clearly incorrect. You may want to inspect several things: 1) Is the IOV modeled and mu referenced in a similar manner as the example 7 in the intro7.pdf manual 2) Based on your GRD setting, it appears that your original, non-IOV code has Thetas 10 and 11 as sigma-like parameters. Is this still the case with the new IOV control stream Robert J. Bauer, Ph.D. Vice President, Pharmacometrics ICON Development Solutions Tel: (215) 616-6428 Mob: (925) 286-0769 Email: [email protected] Web: www.icondevsolutions.com

Hi, Thuy shows an example of using IMP to calculate OBJ with a value of NITER=300. In the intro7.pdf (p 16) there is this example with NITER=5: "After the analysis, suitable objective function for hypothesis testing and standard errors can be obtained by importance sampling at the final population parameter values. Thus, one could issue this sequence of commands: $EST METHOD=SAEM NBURN=2000 NITER=1000 $EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5" On p 30 there is an example with NITER=10: "Importance sampling for 10 iterations, expectation step only (this evaluates OBJF without moving population parameters). Note that SIGL=7 that was set for the previous $EST command is assumed for this $EST command as well $EST METHOD=IMP INTERACTION ISAMPLE=1000 NITER=10 EONLY=1" And on p 31 there is another example which now has NITER=5 again. "$EST METHOD=SAEM NBURN=200 NITER=500 PRINT=100 ; Calculate OBJF by importance sampling $EST METHOD=IMP EONLY=1 NITER=5 ISAMPLE=3000 PRINT=1" Intro7.pdf discusses the use of ISAMPLE on p 14 and EONLY and NITER on p15: "ISAMPLE Sets number of random samples per subject used for expectation step (default 300). Usually 300 is sufficient, but may require 1000-3000 for very sparse data, and when desiring objective function evaluation with low Monte Carlo noise. EONLY=1 Evaluate the objective function by performing only the expectation step, without advancing the population parameters (default is 0, population parameters are updated). When this method is used, NITER should equal 5 to 10, to allow proposal density to improve with each iteration, since mean and variance of parameters of normal or t distribution proposal density are obtained from the previous iteration. Also it is good to get several objective function values to assess the Monte Carlo noise in it." I tried one test problem which ran FOCE then SAEM then IMP with different values for NITER in the IMP step e.g. here with NITER=5: $EST METHOD=COND INTER MAX=9990 NSIG=3 SIGL=9 PRINT=20 NOABORT $EST METHOD=SAEM NBURN=500 NITER=200 PRINT=100 $EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5 With NITER=0 and 1 the OBJ was clearly different but with NITER=5 the OBJ was very similar (but not identical) to the other IMP runs and to the FOCE run. As might be expected the run times get longer with larger NITER values so it seems that NITER=5 is an empirical compromise between estimation of OBJ and run time. It also illustrates the "Monte Carlo noise" in the OBJ estimate. NITER for IMP Obj Meth Seconds - 217.997 FOCE 0.91 0 225.403 IMP 6.70 1 223.467 IMP 7.98 5 217.639 IMP 8.36 10 217.017 IMP 10.31 100 217.469 IMP 39.03 500 217.399 IMP 168.08 So, if you have nothing much to do (Thuy <grin>) then set NITER=300 or more but if you want to get the OBJ value more quickly than a value of 5 would seem to be good enough. Best wishes, Nick Vu, Thuy wrote: > Dear NM7 users, > > According to the examples in the users guide, the IMP step followed the SAEM step with EONLY=1 to evaluate the objective function. In all of my models (for the same data set), I have these two $EST steps: $EST METHOD=SAEM INTER FILE=saem.txt NBURN=3000 NITER=2000 PRINT=5 CTYPE=3 GRD=DDDDDDDDDSSDDD SEED=20091013 ISAMPLE=2 $EST METHOD=IMP EONLY=1 FILE=imp.txt NITER=300 ISAMPLE=3000 PRINT=1 I -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +64 21 46 23 53 http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford