distribution assumption of Eta in NONMEM

From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Friday, May 28, 2010 9:08 AM To: [email protected] Subject: [NMusers] distribution assumption of Eta in NONMEM Dear users, Is it true NONMEM dose not assume Eta a normal distribution? If it does not, I wonder what distribution it assumes? I guess this is critical when we do simulations. Thanks -- The information contained in this email message may contain confidential or legally privileged information and is intended solely for the use of the named recipient(s). No confidentiality or privilege is waived or lost by any transmission error. If the reader of this message is not the intended recipient, please immediately delete the e-mail and all copies of it from your system, destroy any hard copies of it and notify the sender either by telephone or return e-mail. Any direct or indirect use, disclosure, distribution, printing, or copying of any part of this message is prohibited. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of XOMA.

From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Friday, May 28, 2010 2:27 PM To: Serge Guzy; [email protected] Subject: Re: [NMusers] distribution assumption of Eta in NONMEM I could not find in the NONMEM help guide that explicitly mentioned a normal distribution is assumed, only it was clearly mentioned of assumption of mean of zero. _____ From: Serge Guzy <[email protected]> To: Ethan Wu <[email protected]>; [email protected] Sent: Fri, May 28, 2010 1:25:24 PM Subject: RE: [NMusers] distribution assumption of Eta in NONMEM As far as I know, this is the assumption in most of the population programs like NONMEM, SADAPT, PDX-MC-PEM and SAEM. Therefore when you simulate, random values from a normal distribution are generated. However, you have the flexibility to use any transformation to create distributions for your model parameters that will depart from pure normality. For example, CL=theta(1)*exp(eta(1)) will generate a log-normal distribution for the clearance although the random deviates are all from the normal distribution. I am not sure how you can simulate data sets if you are using the non parametric option that is indeed available in NONMEM. Serge Guzy; Ph.D President, CEO, POP_PHARM www.poppharm.com http://www.poppharm.com/ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Friday, May 28, 2010 9:08 AM To: [email protected] Subject: [NMusers] distribution assumption of Eta in NONMEM Dear users, Is it true NONMEM dose not assume Eta a normal distribution? If it does not, I wonder what distribution it assumes? I guess this is critical when we do simulations. Thanks _____ The information contained in this email message may contain confidential or legally privileged information and is intended solely for the use of the named recipient(s). No confidentiality or privilege is waived or lost by any transmission error. If the reader of this message is not the intended recipient, please immediately delete the e-mail and all copies of it from your system, destroy any hard copies of it and notify the sender either by telephone or return e-mail. Any direct or indirect use, disclosure, distribution, printing, or copying of any part of this message is prohibited. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of XOMA.

From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Friday, May 28, 2010 3:51 PM To: [email protected] Subject: Re: [NMusers] distribution assumption of Eta in NONMEM For estimation NONMEM estimates one parameter to describe the distribution of random effects -- this is the variance (OMEGA) of the distribution. Thus it makes no explicit assumption that the distribution is normal. AFAIK any distribution has a variance. For simulation NONMEM assumes all etas are normally distributed. If you use OMEGA BLOCK(*) then the distribution is multivariate with covariances but still normal. Nick Ethan Wu wrote: I could not find in the NONMEM help guide that explicitly mentioned a normal distribution is assumed, only it was clearly mentioned of assumption of mean of zero. ________________________________ From: Serge Guzy <[email protected]><mailto:[email protected]> To: Ethan Wu <[email protected]><mailto:[email protected]>; [email protected]<mailto:[email protected]> Sent: Fri, May 28, 2010 1:25:24 PM Subject: RE: [NMusers] distribution assumption of Eta in NONMEM As far as I know, this is the assumption in most of the population programs like NONMEM, SADAPT, PDX-MC-PEM and SAEM. Therefore when you simulate, random values from a normal distribution are generated. However, you have the flexibility to use any transformation to create distributions for your model parameters that will depart from pure normality. For example, CL=theta(1)*exp(eta(1)) will generate a log-normal distribution for the clearance although the random deviates are all from the normal distribution. I am not sure how you can simulate data sets if you are using the non parametric option that is indeed available in NONMEM. Serge Guzy; Ph.D President, CEO, POP_PHARM http://www.poppharm.com/ From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Friday, May 28, 2010 9:08 AM To: [email protected]<mailto:[email protected]> Subject: [NMusers] distribution assumption of Eta in NONMEM Dear users, Is it true NONMEM dose not assume Eta a normal distribution? If it does not, I wonder what distribution it assumes? I guess this is critical when we do simulations. Thanks ________________________________ The information contained in this email message may contain confidential or legally privileged information and is intended solely for the use of the named recipient(s). No confidentiality or privilege is waived or lost by any transmission error. If the reader of this message is not the intended recipient, please immediately delete the e-mail and all copies of it from your system, destroy any hard copies of it and notify the sender either by telephone or return e-mail. Any direct or indirect use, disclosure, distribution, printing, or copying of any part of this message is prohibited. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of XOMA. -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 email: [email protected]<mailto:[email protected]> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

From: [email protected] [mailto:[email protected]] On Behalf Of Jurgen Bulitta Sent: Friday, May 28, 2010 11:50 PM To: '[email protected]' Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Dear Ethan, There may be two aspects to your question, one is on the assumptions of the algorithm and software implementation and one on the use of the models as described by Nick. To my knowledge, the EM algorithm (e.g. MC-PEM) assumes that the etas are multivariate normally distributed. As described in Bob's paper [1] and the underlying theoretical algorithm development work from Alan Schumitzky [2] and others, the EM algorithm obtains the maximum likelihood estimates for the population means and the variance-covariance matrix by calculating the average of the conditional means and the conditional var-cov matrices of the individual subjects (see equations 21 and 22 in [1]). These equations assume that the parameter population density h(theta | mu, Omega) is multivariate normal. The residual error does not need to follow a normal distribution (see page E64 in Bob's paper [1]). Most of the applications of a model are based on simulations which usually explicitly assume a multivariate normal distribution (or some transformation thereof). Therefore, it seems fair to say that for parametric population PK models, most of the inferences are based on the assumption of a multivariate normal distribution of the "etas" at one or more stages. We rarely have enough subjects to assess the appropriateness of this assumption. You would have to go to a full nonparametric algorithm such as NPML, NPAG or Bob Leary's new method in Phoenix to not assume a normal distribution of the "etas". Best wishes Juergen [1] Bauer RJ, Guzy S, Ng C. AAPS J. 2007;9:E60-83. [2] Schumitzky A . EM algorithms and two stage methods in pharmacokinetics population analysis. In: D'Argenio DZ , ed. Advanced Methods of Pharmacokinetic and Pharmacodynamic Systems Analysis. vol. 2. Boston, MA : Kluwer Academic Publishers ; 1995 :145- 160. From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Friday, May 28, 2010 3:51 PM To: [email protected] Subject: Re: [NMusers] distribution assumption of Eta in NONMEM For estimation NONMEM estimates one parameter to describe the distribution of random effects -- this is the variance (OMEGA) of the distribution. Thus it makes no explicit assumption that the distribution is normal. AFAIK any distribution has a variance. For simulation NONMEM assumes all etas are normally distributed. If you use OMEGA BLOCK(*) then the distribution is multivariate with covariances but still normal. Nick Ethan Wu wrote: I could not find in the NONMEM help guide that explicitly mentioned a normal distribution is assumed, only it was clearly mentioned of assumption of mean of zero. _____ From: Serge Guzy <mailto:[email protected]> <[email protected]> To: Ethan Wu <mailto:[email protected]> <[email protected]>; [email protected] Sent: Fri, May 28, 2010 1:25:24 PM Subject: RE: [NMusers] distribution assumption of Eta in NONMEM As far as I know, this is the assumption in most of the population programs like NONMEM, SADAPT, PDX-MC-PEM and SAEM. Therefore when you simulate, random values from a normal distribution are generated. However, you have the flexibility to use any transformation to create distributions for your model parameters that will depart from pure normality. For example, CL=theta(1)*exp(eta(1)) will generate a log-normal distribution for the clearance although the random deviates are all from the normal distribution. I am not sure how you can simulate data sets if you are using the non parametric option that is indeed available in NONMEM. Serge Guzy; Ph.D President, CEO, POP_PHARM www.poppharm.com http://www.poppharm.com/ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Friday, May 28, 2010 9:08 AM To: [email protected] Subject: [NMusers] distribution assumption of Eta in NONMEM Dear users, Is it true NONMEM dose not assume Eta a normal distribution? If it does not, I wonder what distribution it assumes? I guess this is critical when we do simulations. Thanks _____ The information contained in this email message may contain confidential or legally privileged information and is intended solely for the use of the named recipient(s). No confidentiality or privilege is waived or lost by any transmission error. If the reader of this message is not the intended recipient, please immediately delete the e-mail and all copies of it from your system, destroy any hard copies of it and notify the sender either by telephone or return e-mail. Any direct or indirect use, disclosure, distribution, printing, or copying of any part of this message is prohibited. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of XOMA. -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 email: [email protected] http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

From: [email protected] [mailto:[email protected]] On Behalf Of Eleveld, DJ Sent: Sunday, May 30, 2010 1:20 AM To: Nick Holford; [email protected] Cc: Marc Lavielle Subject: RE: [NMusers] distribution assumption of Eta in NONMEM I'd like to interject a slightly different point of view to the distributional assumption question here. When I hear people speak in terms of the “distribution assumptions of some estimation method” I think its easy for people to jump to the conclusion that the normal distribution assumption is just one of many possible, equally justifiable distributional assumptions that could potentially be made. And that if the normal distribution is the “wrong” one then the results from such an estimation method would be “wrong”. This is what I used to think, but now I believe this is wrong and I'd like to help others from wasting as much time thinking along this path, as I have. >From information theory, information is gained when entropy decreases. So if >you have data from some unknown distribution and if you must make some >distribution assumption in order to analyze the data, you should choose the >highest entropy distribution you can. This insures that your initial >assumptions, the ones you do before you actually consider your data, are the >most uninformative you can make. This is the principle of Maximum Entropy >which is related to Principle of Indifference and the Principle of >Insufficient Reason. A normal distribution has the highest entropy of all real-valued distributions that share the same mean and standard deviation. So if you assume your data has some true SD, then the best distribution to assume would be normal distribution. So we should not think of the normal distribution assumption as one of many equally justifiable choices, it is really the “least-bad” assumption we can make when we do not know the true distribution. Even if normal is the “wrong” distribution, it still remains the “best”, by virtue of being the “least-bad”, because it is the most uninformative assumption that can be made (assuming a some finite true variance). In the real-word we never know the true distribution and so it makes sense to always assume a normal distribution unless we have some scientifically justifiable reason to believe that some other distribution assumption would be advantageous. The Cauchy distribution is a different animal though since its has an infinite variance, and is therefore an even weaker assumption than the finite true SD of a normal distribution. It would possibly be even better than a normal distribution because its entropy is even higher (comparing the standard Cauchy and standard normal). It would be very interesting if Cauchy distributions could be used in NONMEM. Actually, the ratio of two N(0,1) random variables is Cauchy distributed. Maybe this property could be used trick NONMEM into making a Cauchy (or nearly-Cauchy) distributed random variable? Douglas Eleveld _____

On a somewhat unrelated issue - there is one part of the estimation process that can be misleading if a normal assumption is made and that is the use of estimated standard errors to compute confidence intervals (CIs). If likelihood profiling (Holford & Peace 1992) or bootstraps (Matthews et al. 2004) are used to obtain CIs then it not uncommon to find the CI is asymmetrical and this cannot be predicted from the asymptotic standard error estimate. Computation of CIs with standard errors typically assumes a normal distribution of the uncertainty and this leads to a misleading impression of the uncertainty that can be only be discovered by methods which do not make this normal assumption. This is not just a problem with NONMEM - it is a problem with any procedure that only provides a standard error as an estimate of uncertainty. Nick Holford, N. H. G. and K. E. Peace (1992). "Results and validation of a population pharmacodynamic model for cognitive effects in Alzheimer patients treated with tacrine." Proceedings of the National Academy of Sciences of the United States of America 89(23): 11471-11475. Matthews, I., C. Kirkpatrick, et al. (2004). "Quantitative justification for target concentration intervention - Parameter variability and predictive performance using population pharmacokinetic models for aminoglycosides." British Journal of Clinical Pharmacology 58(1): 8-19. Eleveld, DJ wrote: > I'd like to interject a slightly different point of view to the distributional assumption question here. When I hear people speak in terms of the “distribution assumptions of some estimation method” I think its easy for people to jump to the conclusion that the normal distribution assumption is just one of many possible, equally justifiable distributional assumptions that could potentially be made. And that if the normal distribution is the “wrong” one then the results from such an estimation method would be “wrong”. This is what I used to think, but now I believe this is wrong and I'd like to help others from wasting as much time thinking along this path, as I have. > > From information theory, information is gained when entropy decreases. So if you have data from some unknown distribution and if you must make some distribution assumption in order to analyze the data, you should choose the highest entropy distribution you can. This insures that your initial assumptions, the ones you do before you actually consider your data, are the most uninformative you can make. This is the principle of Maximum Entropy which is related to Principle of Indifference and the Principle of Insufficient Reason. > > A normal distribution has the highest entropy of all real-valued distributions that share the same mean and standard deviation. So if you assume your data has some true SD, then the best distribution to assume would be normal distribution. So we should not think of the normal distribution assumption as one of many equally justifiable choices, it is really the “least-bad” assumption we can make when we do not know the true distribution. Even if normal is the “wrong” distribution, it still remains the “best”, by virtue of being the “least-bad”, because it is the most uninformative assumption that can be made (assuming a some finite true variance). In the real-word we never know the true distribution and so it makes sense to always assume a normal distribution unless we have some scientifically justifiable reason to believe that some other distribution assumption would be advantageous. The Cauchy distribution is a different animal though since its has an infinite variance, and is therefore an even weaker assumption than the finite true SD of a normal distribution. It would possibly be even better than a normal distribution because its entropy is even higher (comparing the standard Cauchy and standard normal). It would be very interesting if Cauchy distributions could be used in NONMEM. Actually, the ratio of two N(0,1) random variables is Cauchy distributed. Maybe this property could be used trick NONMEM into making a Cauchy (or nearly-Cauchy) distributed random variable? > > Douglas Eleveld > > ------------------------------------------------------------------------ > >

From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Monday, May 31, 2010 6:05 PM To: [email protected] Cc: 'Marc Lavielle' Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Hi, I tried to see with brute force how well NONMEM can produce an empirical Bayes estimate when the ETA used for simulation is uniform. I attempted to stress NONMEM with a non-linear problem (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 compared with the theoretical value of 0.0833. The distribution of 1000 EBEs of ETA(1) looked much more uniform than normal. Thus FOCE show no evidence of normality being imposed on the EBEs. $PROB EBE $INPUT ID DV UNIETA $DATA uni1.csv ; 100 subjects with 1 obs each $THETA 5 ; HILL $OMEGA 0.083333333 ; PPV_HILL = 1/12 $SIGMA 0.000001 FIX ; EPS1 $SIM (1234) (5678 UNIFORM) NSUB=10 $EST METHOD=COND MAX=9990 SIG=3 $PRED IF (ICALL.EQ.4) THEN IF (NEWIND.LE.1) THEN CALL RANDOM(2,R) UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 HILL=THETA(1)*EXP(UNIETA) Y=1.1**HILL/(1.1**HILL+1) ENDIF ELSE HILL=THETA(1)*EXP(ETA(1)) Y=1.1**HILL/(1.1**HILL+1) + EPS(1) ENDIF REP=IREP $TABLE ID REP HILL UNIETA ETA(1) Y ONEHEADER NOPRINT FILE=uni.fit I realized after a bit more thought that my suggestion to transform the eta value for estimation wasn't rational so please ignore that senior moment in my earlier email on this topic. Nick -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 email: [email protected] http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

-----Original Message----- From: owner-nmusers@globomaxnm.com [mailto:owner-nmusers@globomaxnm.com] On Behalf Of Leonid Gibiansky Sent: Monday, May 31, 2010 5:31 PM To: Nick Holford; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Nick, I think, transformation idea is the following: Assume that your (true) model is CL=POPCL*exp(ETAunif) where ETAunif is the random variable with uniform distribution. Assume that you have transformation TRANS that converts normal to uniform. Then ETAunif can be presented (exactly) as ETAunif=TRANS(ETAnorm). Therefore, the true model can be presented (again, exactly) as CL=POPCL*exp(TRANS(ETAnorm)) This model should be used for estimation and according to Mats, should provide you the lowest OF Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Nick Holford wrote: > Leonid, > > The result is what I expected. NONMEM just estimates the variance of the > random effects. It doesn't promise to tell you anything about the > distribution. > > It is indeed bad news for simulation if your simulation relies heavily > on the assumption of a normal distribution and the true distribution is > quite different. > > I think you have to be very careful looking at posthoc ETAs. They are > not informative about the true ETA distribution unless you can be sure > that you have low shrinkage. If shrinkage is not low then a true uniform > will become more normal looking because the tails will collapse. > > The approach that Mats seems to suggest is to try different > transformations of NONMEM's ETA variables to try to lower the OFV. What > is not clear to me is why these transformations which lower the OFV will > make the simulation better when the ETA variables that are used for the > simulation are required to be normally distributed. > > Imagine I use this for estimation: > CL=POPCL*EXP(ETA(1)) where the true ETA is uniform > If I now use the estimated OMEGA(1,1) which will be a good estimate of > the uniform distribution variance, uvar, for simulation then I am using > CL=POPCL*EXP(N(0,uvar)) > which will be wrong because I am now assuming a normal distribution but > using the variance of a uniform. > > Now suppose I try: > CL=POPCL*TRANS(ETA(1)) where TRANS is some transformation that lowers > the OFV to the lowest I can find but the true ETA is still uniform > If I now use the same transformation for simulation with an OMEGA(1,1) > estimate of the variance transvar > CL=POPCL*TRANS(N(0,transvar)) which uses a normal distribution then why > should I expect the simulated distribution of CL to resemble the true > distribution with a uniform ETA? > > Nick > > Leonid Gibiansky wrote: >> Hi Nick, >> I think, I understood it from your original e-mail, but it was so >> unexpected that I asked to confirm it. >> >> Actually, not a good news from your example. >> >> Nonmem cannot distinguish two models: >> with normal distribution, and >> with uniform distributions >> as long as they have the same variance. >> >> So if you simulate from the model, you will end up with very different >> results: either simular to the original data (if by chance, your >> original problem happens to be with normal distribution) or very >> different (if original distribution was uniform). >> >> This shows the need to investigate normality of posthoc ETAs very >> carefully. >> >> Very interesting example >> Thanks >> Leonid >> >> -------------------------------------- >> Leonid Gibiansky, Ph.D. >> President, QuantPharm LLC >> web: www.quantpharm.com >> e-mail: LGibiansky at quantpharm.com >> tel: (301) 767 5566 >> >> >> >> >> Nick Holford wrote: >>> Leonid, >>> >>> I meant by OMEGA(1) the OMEGA value estimated by NONMEM. I suppose I >>> should have written OMEGA(1,1) to be more precise -- sorry! >>> >>> Nick >>> >>> Leonid Gibiansky wrote: >>>> Nick, Mats >>>> >>>> I would guess that nonmem should inflate variance (for this example) >>>> trying to fit the observed uniform (-0.5, 0.5) into some normal N(0, >>>> ?). This example (if I read it correctly) shows that Nonmem somehow >>>> estimates variance without making distribution assumption. >>>> Nick, you mentioned: >>>> >>>> "the mean estimate of OMEGA(1) was 0.0827" >>>> >>>> does it mean that Nonmem-estimated OMEGA was close to 0.0827 or you >>>> refer to the variances of estimated ETAs? >>>> >>>> Thanks >>>> Leonid >>>> >>>> >>>> -------------------------------------- >>>> Leonid Gibiansky, Ph.D. >>>> President, QuantPharm LLC >>>> web: www.quantpharm.com >>>> e-mail: LGibiansky at quantpharm.com >>>> tel: (301) 767 5566 >>>> >>>> >>>> >>>> >>>> Mats Karlsson wrote: >>>>> Nick, >>>>> >>>>> >>>>> >>>>> It has been showed over and over again that empirical Bayes >>>>> estimates, when individual data is rich, will resemble the true >>>>> individual parameter regardless of the underlying distribution. >>>>> Therefore I don’t understand what you think this exercise contributes. >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Mats >>>>> >>>>> >>>>> >>>>> Mats Karlsson, PhD >>>>> >>>>> Professor of Pharmacometrics >>>>> >>>>> Dept of Pharmaceutical Biosciences >>>>> >>>>> Uppsala University >>>>> >>>>> Box 591 >>>>> >>>>> 751 24 Uppsala Sweden >>>>> >>>>> phone: +46 18 4714105 >>>>> >>>>> fax: +46 18 471 4003 >>>>> >>>>> >>>>> >>>>> *From:* owner-nmusers@globomaxnm.com >>>>> [mailto:owner-nmusers@globomaxnm.com] *On Behalf Of *Nick Holford >>>>> *Sent:* Monday, May 31, 2010 6:05 PM >>>>> *To:* nmusers globomaxnm.com >>>>> *Cc:* 'Marc Lavielle' >>>>> *Subject:* Re: [NMusers] distribution assumption of Eta in NONMEM >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I tried to see with brute force how well NONMEM can produce an >>>>> empirical Bayes estimate when the ETA used for simulation is >>>>> uniform. I attempted to stress NONMEM with a non-linear problem >>>>> (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 >>>>> compared with the theoretical value of 0.0833. >>>>> >>>>> The distribution of 1000 EBEs of ETA(1) looked much more uniform >>>>> than normal. >>>>> Thus FOCE show no evidence of normality being imposed on the EBEs. >>>>> >>>>> $PROB EBE >>>>> $INPUT ID DV UNIETA >>>>> $DATA uni1.csv ; 100 subjects with 1 obs each >>>>> $THETA 5 ; HILL >>>>> $OMEGA 0.083333333 ; PPV_HILL = 1/12 >>>>> $SIGMA 0.000001 FIX ; EPS1 >>>>> >>>>> $SIM (1234) (5678 UNIFORM) NSUB=10 >>>>> $EST METHOD=COND MAX=9990 SIG=3 >>>>> $PRED >>>>> IF (ICALL.EQ.4) THEN >>>>> IF (NEWIND.LE.1) THEN >>>>> CALL RANDOM(2,R) >>>>> UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 >>>>> HILL=THETA(1)*EXP(UNIETA) >>>>> Y=1.1**HILL/(1.1**HILL+1) >>>>> ENDIF >>>>> ELSE >>>>> >>>>> HILL=THETA(1)*EXP(ETA(1)) >>>>> Y=1.1**HILL/(1.1**HILL+1) + EPS(1) >>>>> ENDIF >>>>> >>>>> REP=IREP >>>>> >>>>> $TABLE ID REP HILL UNIETA ETA(1) Y >>>>> ONEHEADER NOPRINT FILE=uni.fit >>>>> >>>>> I realized after a bit more thought that my suggestion to transform >>>>> the eta value for estimation wasn't rational so please ignore that >>>>> senior moment in my earlier email on this topic. >>>>> >>>>> Nick >>>>> >>>>> >>>>> -- >>>>> >>>>> Nick Holford, Professor Clinical Pharmacology >>>>> >>>>> Dept Pharmacology & Clinical Pharmacology >>>>> >>>>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New >>>>> Zealand >>>>> >>>>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>>>> >>>>> email: n.holford@auckland.ac.nz <mailto:n.holford@auckland.ac.nz> >>>>> >>>>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>>>> >>> >>> -- >>> Nick Holford, Professor Clinical Pharmacology >>> Dept Pharmacology & Clinical Pharmacology >>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand >>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>> email: n.holford >>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>> > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 > email: n.holford > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >

-----Original Message----- From: owner-nmusers On Behalf Of Leonid Gibiansky Sent: Monday, 31 May, 2010 23:31 To: Nick Holford; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Nick, I think, transformation idea is the following: Assume that your (true) model is CL=POPCL*exp(ETAunif) where ETAunif is the random variable with uniform distribution. Assume that you have transformation TRANS that converts normal to uniform. Then ETAunif can be presented (exactly) as ETAunif=TRANS(ETAnorm). Therefore, the true model can be presented (again, exactly) as CL=POPCL*exp(TRANS(ETAnorm)) This model should be used for estimation and according to Mats, should provide you the lowest OF Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Nick Holford wrote: > Leonid, > > The result is what I expected. NONMEM just estimates the variance of > the random effects. It doesn't promise to tell you anything about the > distribution. > > It is indeed bad news for simulation if your simulation relies heavily > on the assumption of a normal distribution and the true distribution > is quite different. > > I think you have to be very careful looking at posthoc ETAs. They are > not informative about the true ETA distribution unless you can be sure > that you have low shrinkage. If shrinkage is not low then a true > uniform will become more normal looking because the tails will collapse. > > The approach that Mats seems to suggest is to try different > transformations of NONMEM's ETA variables to try to lower the OFV. > What is not clear to me is why these transformations which lower the > OFV will make the simulation better when the ETA variables that are > used for the simulation are required to be normally distributed. > > Imagine I use this for estimation: > CL=POPCL*EXP(ETA(1)) where the true ETA is uniform If I now use the > estimated OMEGA(1,1) which will be a good estimate of the uniform > distribution variance, uvar, for simulation then I am using > CL=POPCL*EXP(N(0,uvar)) > which will be wrong because I am now assuming a normal distribution > but using the variance of a uniform. > > Now suppose I try: > CL=POPCL*TRANS(ETA(1)) where TRANS is some transformation that lowers > the OFV to the lowest I can find but the true ETA is still uniform If > I now use the same transformation for simulation with an OMEGA(1,1) > estimate of the variance transvar > CL=POPCL*TRANS(N(0,transvar)) which uses a normal distribution then > why should I expect the simulated distribution of CL to resemble the > true distribution with a uniform ETA? > > Nick > > Leonid Gibiansky wrote: >> Hi Nick, >> I think, I understood it from your original e-mail, but it was so >> unexpected that I asked to confirm it. >> >> Actually, not a good news from your example. >> >> Nonmem cannot distinguish two models: >> with normal distribution, and >> with uniform distributions >> as long as they have the same variance. >> >> So if you simulate from the model, you will end up with very >> different >> results: either simular to the original data (if by chance, your >> original problem happens to be with normal distribution) or very >> different (if original distribution was uniform). >> >> This shows the need to investigate normality of posthoc ETAs very >> carefully. >> >> Very interesting example >> Thanks >> Leonid >> >> -------------------------------------- >> Leonid Gibiansky, Ph.D. >> President, QuantPharm LLC >> web: www.quantpharm.com >> e-mail: LGibiansky at quantpharm.com >> tel: (301) 767 5566 >> >> >> >> >> Nick Holford wrote: >>> Leonid, >>> >>> I meant by OMEGA(1) the OMEGA value estimated by NONMEM. I suppose I >>> should have written OMEGA(1,1) to be more precise -- sorry! >>> >>> Nick >>> >>> Leonid Gibiansky wrote: >>>> Nick, Mats >>>> >>>> I would guess that nonmem should inflate variance (for this >>>> example) trying to fit the observed uniform (-0.5, 0.5) into some >>>> normal N(0, ?). This example (if I read it correctly) shows that >>>> Nonmem somehow estimates variance without making distribution assumption. >>>> Nick, you mentioned: >>>> >>>> "the mean estimate of OMEGA(1) was 0.0827" >>>> >>>> does it mean that Nonmem-estimated OMEGA was close to 0.0827 or you >>>> refer to the variances of estimated ETAs? >>>> >>>> Thanks >>>> Leonid >>>> >>>> >>>> -------------------------------------- >>>> Leonid Gibiansky, Ph.D. >>>> President, QuantPharm LLC >>>> web: www.quantpharm.com >>>> e-mail: LGibiansky at quantpharm.com >>>> tel: (301) 767 5566 >>>> >>>> >>>> >>>> >>>> Mats Karlsson wrote: >>>>> Nick, >>>>> >>>>> >>>>> >>>>> It has been showed over and over again that empirical Bayes >>>>> estimates, when individual data is rich, will resemble the true >>>>> individual parameter regardless of the underlying distribution. >>>>> Therefore I don't understand what you think this exercise contributes. >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Mats >>>>> >>>>> >>>>> >>>>> Mats Karlsson, PhD >>>>> >>>>> Professor of Pharmacometrics >>>>> >>>>> Dept of Pharmaceutical Biosciences >>>>> >>>>> Uppsala University >>>>> >>>>> Box 591 >>>>> >>>>> 751 24 Uppsala Sweden >>>>> >>>>> phone: +46 18 4714105 >>>>> >>>>> fax: +46 18 471 4003 >>>>> >>>>> >>>>> >>>>> *From:* owner-nmusers >>>>> [mailto:owner-nmusers >>>>> *Sent:* Monday, May 31, 2010 6:05 PM >>>>> *To:* nmusers >>>>> *Cc:* 'Marc Lavielle' >>>>> *Subject:* Re: [NMusers] distribution assumption of Eta in NONMEM >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I tried to see with brute force how well NONMEM can produce an >>>>> empirical Bayes estimate when the ETA used for simulation is >>>>> uniform. I attempted to stress NONMEM with a non-linear problem >>>>> (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 >>>>> compared with the theoretical value of 0.0833. >>>>> >>>>> The distribution of 1000 EBEs of ETA(1) looked much more uniform >>>>> than normal. >>>>> Thus FOCE show no evidence of normality being imposed on the EBEs. >>>>> >>>>> $PROB EBE >>>>> $INPUT ID DV UNIETA >>>>> $DATA uni1.csv ; 100 subjects with 1 obs each $THETA 5 ; HILL >>>>> $OMEGA 0.083333333 ; PPV_HILL = 1/12 $SIGMA 0.000001 FIX ; EPS1 >>>>> >>>>> $SIM (1234) (5678 UNIFORM) NSUB $EST METHOD=COND MAX90 SIG=3 >>>>> $PRED IF (ICALL.EQ.4) THEN >>>>> IF (NEWIND.LE.1) THEN >>>>> CALL RANDOM(2,R) >>>>> UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 >>>>> HILL=THETA(1)*EXP(UNIETA) >>>>> Y=1.1**HILL/(1.1**HILL+1) >>>>> ENDIF >>>>> ELSE >>>>> >>>>> HILL=THETA(1)*EXP(ETA(1)) >>>>> Y=1.1**HILL/(1.1**HILL+1) + EPS(1) ENDIF >>>>> >>>>> REP=IREP >>>>> >>>>> $TABLE ID REP HILL UNIETA ETA(1) Y ONEHEADER NOPRINT FILE=uni.fit >>>>> >>>>> I realized after a bit more thought that my suggestion to >>>>> transform the eta value for estimation wasn't rational so please >>>>> ignore that senior moment in my earlier email on this topic. >>>>> >>>>> Nick >>>>> >>>>> >>>>> -- >>>>> >>>>> Nick Holford, Professor Clinical Pharmacology >>>>> >>>>> Dept Pharmacology & Clinical Pharmacology >>>>> >>>>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New >>>>> Zealand >>>>> >>>>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>>>> >>>>> email: n.holford >>>>> >>>>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>>>> >>> >>> -- >>> Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & >>> Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag >>> 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 >>> mobile:+64(21)46 23 53 >>> email: n.holford >>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>> > > -- > Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & > Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag > 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 > mobile:+64(21)46 23 53 > email: n.holford > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited --- Please immediately and permanently delete.

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Leonid Gibiansky Sent: Monday, May 31, 2010 5:31 PM To: Nick Holford; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Nick, I think, transformation idea is the following: Assume that your (true) model is CL=POPCL*exp(ETAunif) where ETAunif is the random variable with uniform distribution. Assume that you have transformation TRANS that converts normal to uniform. Then ETAunif can be presented (exactly) as ETAunif=TRANS(ETAnorm). Therefore, the true model can be presented (again, exactly) as CL=POPCL*exp(TRANS(ETAnorm)) This model should be used for estimation and according to Mats, should provide you the lowest OF Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Nick Holford wrote: > Leonid, > > The result is what I expected. NONMEM just estimates the variance of the > random effects. It doesn't promise to tell you anything about the > distribution. > > It is indeed bad news for simulation if your simulation relies heavily > on the assumption of a normal distribution and the true distribution is > quite different. > > I think you have to be very careful looking at posthoc ETAs. They are > not informative about the true ETA distribution unless you can be sure > that you have low shrinkage. If shrinkage is not low then a true uniform > will become more normal looking because the tails will collapse. > > The approach that Mats seems to suggest is to try different > transformations of NONMEM's ETA variables to try to lower the OFV. What > is not clear to me is why these transformations which lower the OFV will > make the simulation better when the ETA variables that are used for the > simulation are required to be normally distributed. > > Imagine I use this for estimation: > CL=POPCL*EXP(ETA(1)) where the true ETA is uniform > If I now use the estimated OMEGA(1,1) which will be a good estimate of > the uniform distribution variance, uvar, for simulation then I am using > CL=POPCL*EXP(N(0,uvar)) > which will be wrong because I am now assuming a normal distribution but > using the variance of a uniform. > > Now suppose I try: > CL=POPCL*TRANS(ETA(1)) where TRANS is some transformation that lowers > the OFV to the lowest I can find but the true ETA is still uniform > If I now use the same transformation for simulation with an OMEGA(1,1) > estimate of the variance transvar > CL=POPCL*TRANS(N(0,transvar)) which uses a normal distribution then why > should I expect the simulated distribution of CL to resemble the true > distribution with a uniform ETA? > > Nick > > Leonid Gibiansky wrote: >> Hi Nick, >> I think, I understood it from your original e-mail, but it was so >> unexpected that I asked to confirm it. >> >> Actually, not a good news from your example. >> >> Nonmem cannot distinguish two models: >> with normal distribution, and >> with uniform distributions >> as long as they have the same variance. >> >> So if you simulate from the model, you will end up with very different >> results: either simular to the original data (if by chance, your >> original problem happens to be with normal distribution) or very >> different (if original distribution was uniform). >> >> This shows the need to investigate normality of posthoc ETAs very >> carefully. >> >> Very interesting example >> Thanks >> Leonid >> >> -------------------------------------- >> Leonid Gibiansky, Ph.D. >> President, QuantPharm LLC >> web: www.quantpharm.com >> e-mail: LGibiansky at quantpharm.com >> tel: (301) 767 5566 >> >> >> >> >> Nick Holford wrote: >>> Leonid, >>> >>> I meant by OMEGA(1) the OMEGA value estimated by NONMEM. I suppose I >>> should have written OMEGA(1,1) to be more precise -- sorry! >>> >>> Nick >>> >>> Leonid Gibiansky wrote: >>>> Nick, Mats >>>> >>>> I would guess that nonmem should inflate variance (for this example) >>>> trying to fit the observed uniform (-0.5, 0.5) into some normal N(0, >>>> ?). This example (if I read it correctly) shows that Nonmem somehow >>>> estimates variance without making distribution assumption. >>>> Nick, you mentioned: >>>> >>>> "the mean estimate of OMEGA(1) was 0.0827" >>>> >>>> does it mean that Nonmem-estimated OMEGA was close to 0.0827 or you >>>> refer to the variances of estimated ETAs? >>>> >>>> Thanks >>>> Leonid >>>> >>>> >>>> -------------------------------------- >>>> Leonid Gibiansky, Ph.D. >>>> President, QuantPharm LLC >>>> web: www.quantpharm.com >>>> e-mail: LGibiansky at quantpharm.com >>>> tel: (301) 767 5566 >>>> >>>> >>>> >>>> >>>> Mats Karlsson wrote: >>>>> Nick, >>>>> >>>>> >>>>> >>>>> It has been showed over and over again that empirical Bayes >>>>> estimates, when individual data is rich, will resemble the true >>>>> individual parameter regardless of the underlying distribution. >>>>> Therefore I don’t understand what you think this exercise contributes. >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Mats >>>>> >>>>> >>>>> >>>>> Mats Karlsson, PhD >>>>> >>>>> Professor of Pharmacometrics >>>>> >>>>> Dept of Pharmaceutical Biosciences >>>>> >>>>> Uppsala University >>>>> >>>>> Box 591 >>>>> >>>>> 751 24 Uppsala Sweden >>>>> >>>>> phone: +46 18 4714105 >>>>> >>>>> fax: +46 18 471 4003 >>>>> >>>>> >>>>> >>>>> *From:* [email protected] >>>>> [mailto:[email protected]] *On Behalf Of *Nick Holford >>>>> *Sent:* Monday, May 31, 2010 6:05 PM >>>>> *To:* [email protected] >>>>> *Cc:* 'Marc Lavielle' >>>>> *Subject:* Re: [NMusers] distribution assumption of Eta in NONMEM >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I tried to see with brute force how well NONMEM can produce an >>>>> empirical Bayes estimate when the ETA used for simulation is >>>>> uniform. I attempted to stress NONMEM with a non-linear problem >>>>> (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 >>>>> compared with the theoretical value of 0.0833. >>>>> >>>>> The distribution of 1000 EBEs of ETA(1) looked much more uniform >>>>> than normal. >>>>> Thus FOCE show no evidence of normality being imposed on the EBEs. >>>>> >>>>> $PROB EBE >>>>> $INPUT ID DV UNIETA >>>>> $DATA uni1.csv ; 100 subjects with 1 obs each >>>>> $THETA 5 ; HILL >>>>> $OMEGA 0.083333333 ; PPV_HILL = 1/12 >>>>> $SIGMA 0.000001 FIX ; EPS1 >>>>> >>>>> $SIM (1234) (5678 UNIFORM) NSUB=10 >>>>> $EST METHOD=COND MAX=9990 SIG=3 >>>>> $PRED >>>>> IF (ICALL.EQ.4) THEN >>>>> IF (NEWIND.LE.1) THEN >>>>> CALL RANDOM(2,R) >>>>> UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 >>>>> HILL=THETA(1)*EXP(UNIETA) >>>>> Y=1.1**HILL/(1.1**HILL+1) >>>>> ENDIF >>>>> ELSE >>>>> >>>>> HILL=THETA(1)*EXP(ETA(1)) >>>>> Y=1.1**HILL/(1.1**HILL+1) + EPS(1) >>>>> ENDIF >>>>> >>>>> REP=IREP >>>>> >>>>> $TABLE ID REP HILL UNIETA ETA(1) Y >>>>> ONEHEADER NOPRINT FILE=uni.fit >>>>> >>>>> I realized after a bit more thought that my suggestion to transform >>>>> the eta value for estimation wasn't rational so please ignore that >>>>> senior moment in my earlier email on this topic. >>>>> >>>>> Nick >>>>> >>>>> >>>>> -- >>>>> >>>>> Nick Holford, Professor Clinical Pharmacology >>>>> >>>>> Dept Pharmacology & Clinical Pharmacology >>>>> >>>>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New >>>>> Zealand >>>>> >>>>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>>>> >>>>> email: [email protected] <mailto:[email protected]> >>>>> >>>>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>>>> >>> >>> -- >>> Nick Holford, Professor Clinical Pharmacology >>> Dept Pharmacology & Clinical Pharmacology >>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand >>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>> email: [email protected] >>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>> > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 > email: [email protected] > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Michael Fossler Sent: Tuesday, June 01, 2010 3:54 AM To: Leonid Gibiansky; Nick Holford; nmusers Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Interesting topic. Can anyone provide specific transformations of ETAs that they have found useful? Mike Fossler GSK -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Leonid Gibiansky Sent: Monday, May 31, 2010 5:31 PM To: Nick Holford; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Nick, I think, transformation idea is the following: Assume that your (true) model is CL=POPCL*exp(ETAunif) where ETAunif is the random variable with uniform distribution. Assume that you have transformation TRANS that converts normal to uniform. Then ETAunif can be presented (exactly) as ETAunif=TRANS(ETAnorm). Therefore, the true model can be presented (again, exactly) as CL=POPCL*exp(TRANS(ETAnorm)) This model should be used for estimation and according to Mats, should provide you the lowest OF Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Nick Holford wrote: > Leonid, > > The result is what I expected. NONMEM just estimates the variance of the > random effects. It doesn't promise to tell you anything about the > distribution. > > It is indeed bad news for simulation if your simulation relies heavily > on the assumption of a normal distribution and the true distribution is > quite different. > > I think you have to be very careful looking at posthoc ETAs. They are > not informative about the true ETA distribution unless you can be sure > that you have low shrinkage. If shrinkage is not low then a true uniform > will become more normal looking because the tails will collapse. > > The approach that Mats seems to suggest is to try different > transformations of NONMEM's ETA variables to try to lower the OFV. What > is not clear to me is why these transformations which lower the OFV will > make the simulation better when the ETA variables that are used for the > simulation are required to be normally distributed. > > Imagine I use this for estimation: > CL=POPCL*EXP(ETA(1)) where the true ETA is uniform > If I now use the estimated OMEGA(1,1) which will be a good estimate of > the uniform distribution variance, uvar, for simulation then I am using > CL=POPCL*EXP(N(0,uvar)) > which will be wrong because I am now assuming a normal distribution but > using the variance of a uniform. > > Now suppose I try: > CL=POPCL*TRANS(ETA(1)) where TRANS is some transformation that lowers > the OFV to the lowest I can find but the true ETA is still uniform > If I now use the same transformation for simulation with an OMEGA(1,1) > estimate of the variance transvar > CL=POPCL*TRANS(N(0,transvar)) which uses a normal distribution then why > should I expect the simulated distribution of CL to resemble the true > distribution with a uniform ETA? > > Nick > > Leonid Gibiansky wrote: >> Hi Nick, >> I think, I understood it from your original e-mail, but it was so >> unexpected that I asked to confirm it. >> >> Actually, not a good news from your example. >> >> Nonmem cannot distinguish two models: >> with normal distribution, and >> with uniform distributions >> as long as they have the same variance. >> >> So if you simulate from the model, you will end up with very different >> results: either simular to the original data (if by chance, your >> original problem happens to be with normal distribution) or very >> different (if original distribution was uniform). >> >> This shows the need to investigate normality of posthoc ETAs very >> carefully. >> >> Very interesting example >> Thanks >> Leonid >> >> -------------------------------------- >> Leonid Gibiansky, Ph.D. >> President, QuantPharm LLC >> web: www.quantpharm.com >> e-mail: LGibiansky at quantpharm.com >> tel: (301) 767 5566 >> >> >> >> >> Nick Holford wrote: >>> Leonid, >>> >>> I meant by OMEGA(1) the OMEGA value estimated by NONMEM. I suppose I >>> should have written OMEGA(1,1) to be more precise -- sorry! >>> >>> Nick >>> >>> Leonid Gibiansky wrote: >>>> Nick, Mats >>>> >>>> I would guess that nonmem should inflate variance (for this example) >>>> trying to fit the observed uniform (-0.5, 0.5) into some normal N(0, >>>> ?). This example (if I read it correctly) shows that Nonmem somehow >>>> estimates variance without making distribution assumption. >>>> Nick, you mentioned: >>>> >>>> "the mean estimate of OMEGA(1) was 0.0827" >>>> >>>> does it mean that Nonmem-estimated OMEGA was close to 0.0827 or you >>>> refer to the variances of estimated ETAs? >>>> >>>> Thanks >>>> Leonid >>>> >>>> >>>> -------------------------------------- >>>> Leonid Gibiansky, Ph.D. >>>> President, QuantPharm LLC >>>> web: www.quantpharm.com >>>> e-mail: LGibiansky at quantpharm.com >>>> tel: (301) 767 5566 >>>> >>>> >>>> >>>> >>>> Mats Karlsson wrote: >>>>> Nick, >>>>> >>>>> >>>>> >>>>> It has been showed over and over again that empirical Bayes >>>>> estimates, when individual data is rich, will resemble the true >>>>> individual parameter regardless of the underlying distribution. >>>>> Therefore I don’t understand what you think this exercise contributes. >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Mats >>>>> >>>>> >>>>> >>>>> Mats Karlsson, PhD >>>>> >>>>> Professor of Pharmacometrics >>>>> >>>>> Dept of Pharmaceutical Biosciences >>>>> >>>>> Uppsala University >>>>> >>>>> Box 591 >>>>> >>>>> 751 24 Uppsala Sweden >>>>> >>>>> phone: +46 18 4714105 >>>>> >>>>> fax: +46 18 471 4003 >>>>> >>>>> >>>>> >>>>> *From:* [email protected] >>>>> [mailto:[email protected]] *On Behalf Of *Nick Holford >>>>> *Sent:* Monday, May 31, 2010 6:05 PM >>>>> *To:* [email protected] >>>>> *Cc:* 'Marc Lavielle' >>>>> *Subject:* Re: [NMusers] distribution assumption of Eta in NONMEM >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I tried to see with brute force how well NONMEM can produce an >>>>> empirical Bayes estimate when the ETA used for simulation is >>>>> uniform. I attempted to stress NONMEM with a non-linear problem >>>>> (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 >>>>> compared with the theoretical value of 0.0833. >>>>> >>>>> The distribution of 1000 EBEs of ETA(1) looked much more uniform >>>>> than normal. >>>>> Thus FOCE show no evidence of normality being imposed on the EBEs. >>>>> >>>>> $PROB EBE >>>>> $INPUT ID DV UNIETA >>>>> $DATA uni1.csv ; 100 subjects with 1 obs each >>>>> $THETA 5 ; HILL >>>>> $OMEGA 0.083333333 ; PPV_HILL = 1/12 >>>>> $SIGMA 0.000001 FIX ; EPS1 >>>>> >>>>> $SIM (1234) (5678 UNIFORM) NSUB=10 >>>>> $EST METHOD=COND MAX=9990 SIG=3 >>>>> $PRED >>>>> IF (ICALL.EQ.4) THEN >>>>> IF (NEWIND.LE.1) THEN >>>>> CALL RANDOM(2,R) >>>>> UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 >>>>> HILL=THETA(1)*EXP(UNIETA) >>>>> Y=1.1**HILL/(1.1**HILL+1) >>>>> ENDIF >>>>> ELSE >>>>> >>>>> HILL=THETA(1)*EXP(ETA(1)) >>>>> Y=1.1**HILL/(1.1**HILL+1) + EPS(1) >>>>> ENDIF >>>>> >>>>> REP=IREP >>>>> >>>>> $TABLE ID REP HILL UNIETA ETA(1) Y >>>>> ONEHEADER NOPRINT FILE=uni.fit >>>>> >>>>> I realized after a bit more thought that my suggestion to transform >>>>> the eta value for estimation wasn't rational so please ignore that >>>>> senior moment in my earlier email on this topic. >>>>> >>>>> Nick >>>>> >>>>> >>>>> -- >>>>> >>>>> Nick Holford, Professor Clinical Pharmacology >>>>> >>>>> Dept Pharmacology & Clinical Pharmacology >>>>> >>>>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New >>>>> Zealand >>>>> >>>>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>>>> >>>>> email: [email protected] <mailto:[email protected]> >>>>> >>>>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>>>> >>> >>> -- >>> Nick Holford, Professor Clinical Pharmacology >>> Dept Pharmacology & Clinical Pharmacology >>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand >>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>> email: [email protected] >>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>> > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 > email: [email protected] > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Michael Fossler Sent: Tuesday, 01 June, 2010 3:54 To: Leonid Gibiansky; Nick Holford; nmusers Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Interesting topic. Can anyone provide specific transformations of ETAs that they have found useful? Mike Fossler GSK -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Leonid Gibiansky Sent: Monday, May 31, 2010 5:31 PM To: Nick Holford; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Nick, I think, transformation idea is the following: Assume that your (true) model is CL=POPCL*exp(ETAunif) where ETAunif is the random variable with uniform distribution. Assume that you have transformation TRANS that converts normal to uniform. Then ETAunif can be presented (exactly) as ETAunif=TRANS(ETAnorm). Therefore, the true model can be presented (again, exactly) as CL=POPCL*exp(TRANS(ETAnorm)) This model should be used for estimation and according to Mats, should provide you the lowest OF Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Nick Holford wrote: > Leonid, > > The result is what I expected. NONMEM just estimates the variance of > the random effects. It doesn't promise to tell you anything about the > distribution. > > It is indeed bad news for simulation if your simulation relies heavily > on the assumption of a normal distribution and the true distribution > is quite different. > > I think you have to be very careful looking at posthoc ETAs. They are > not informative about the true ETA distribution unless you can be sure > that you have low shrinkage. If shrinkage is not low then a true > uniform will become more normal looking because the tails will collapse. > > The approach that Mats seems to suggest is to try different > transformations of NONMEM's ETA variables to try to lower the OFV. > What is not clear to me is why these transformations which lower the > OFV will make the simulation better when the ETA variables that are > used for the simulation are required to be normally distributed. > > Imagine I use this for estimation: > CL=POPCL*EXP(ETA(1)) where the true ETA is uniform If I now use the > estimated OMEGA(1,1) which will be a good estimate of the uniform > distribution variance, uvar, for simulation then I am using > CL=POPCL*EXP(N(0,uvar)) > which will be wrong because I am now assuming a normal distribution > but using the variance of a uniform. > > Now suppose I try: > CL=POPCL*TRANS(ETA(1)) where TRANS is some transformation that lowers > the OFV to the lowest I can find but the true ETA is still uniform If > I now use the same transformation for simulation with an OMEGA(1,1) > estimate of the variance transvar > CL=POPCL*TRANS(N(0,transvar)) which uses a normal distribution then > why should I expect the simulated distribution of CL to resemble the > true distribution with a uniform ETA? > > Nick > > Leonid Gibiansky wrote: >> Hi Nick, >> I think, I understood it from your original e-mail, but it was so >> unexpected that I asked to confirm it. >> >> Actually, not a good news from your example. >> >> Nonmem cannot distinguish two models: >> with normal distribution, and >> with uniform distributions >> as long as they have the same variance. >> >> So if you simulate from the model, you will end up with very >> different >> results: either simular to the original data (if by chance, your >> original problem happens to be with normal distribution) or very >> different (if original distribution was uniform). >> >> This shows the need to investigate normality of posthoc ETAs very >> carefully. >> >> Very interesting example >> Thanks >> Leonid >> >> -------------------------------------- >> Leonid Gibiansky, Ph.D. >> President, QuantPharm LLC >> web: www.quantpharm.com >> e-mail: LGibiansky at quantpharm.com >> tel: (301) 767 5566 >> >> >> >> >> Nick Holford wrote: >>> Leonid, >>> >>> I meant by OMEGA(1) the OMEGA value estimated by NONMEM. I suppose I >>> should have written OMEGA(1,1) to be more precise -- sorry! >>> >>> Nick >>> >>> Leonid Gibiansky wrote: >>>> Nick, Mats >>>> >>>> I would guess that nonmem should inflate variance (for this >>>> example) trying to fit the observed uniform (-0.5, 0.5) into some >>>> normal N(0, ?). This example (if I read it correctly) shows that >>>> Nonmem somehow estimates variance without making distribution assumption. >>>> Nick, you mentioned: >>>> >>>> "the mean estimate of OMEGA(1) was 0.0827" >>>> >>>> does it mean that Nonmem-estimated OMEGA was close to 0.0827 or you >>>> refer to the variances of estimated ETAs? >>>> >>>> Thanks >>>> Leonid >>>> >>>> >>>> -------------------------------------- >>>> Leonid Gibiansky, Ph.D. >>>> President, QuantPharm LLC >>>> web: www.quantpharm.com >>>> e-mail: LGibiansky at quantpharm.com >>>> tel: (301) 767 5566 >>>> >>>> >>>> >>>> >>>> Mats Karlsson wrote: >>>>> Nick, >>>>> >>>>> >>>>> >>>>> It has been showed over and over again that empirical Bayes >>>>> estimates, when individual data is rich, will resemble the true >>>>> individual parameter regardless of the underlying distribution. >>>>> Therefore I don't understand what you think this exercise contributes. >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Mats >>>>> >>>>> >>>>> >>>>> Mats Karlsson, PhD >>>>> >>>>> Professor of Pharmacometrics >>>>> >>>>> Dept of Pharmaceutical Biosciences >>>>> >>>>> Uppsala University >>>>> >>>>> Box 591 >>>>> >>>>> 751 24 Uppsala Sweden >>>>> >>>>> phone: +46 18 4714105 >>>>> >>>>> fax: +46 18 471 4003 >>>>> >>>>> >>>>> >>>>> *From:* [email protected] >>>>> [mailto:[email protected]] *On Behalf Of *Nick Holford >>>>> *Sent:* Monday, May 31, 2010 6:05 PM >>>>> *To:* [email protected] >>>>> *Cc:* 'Marc Lavielle' >>>>> *Subject:* Re: [NMusers] distribution assumption of Eta in NONMEM >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I tried to see with brute force how well NONMEM can produce an >>>>> empirical Bayes estimate when the ETA used for simulation is >>>>> uniform. I attempted to stress NONMEM with a non-linear problem >>>>> (the average DV is 0.62). The mean estimate of OMEGA(1) was 0.0827 >>>>> compared with the theoretical value of 0.0833. >>>>> >>>>> The distribution of 1000 EBEs of ETA(1) looked much more uniform >>>>> than normal. >>>>> Thus FOCE show no evidence of normality being imposed on the EBEs. >>>>> >>>>> $PROB EBE >>>>> $INPUT ID DV UNIETA >>>>> $DATA uni1.csv ; 100 subjects with 1 obs each $THETA 5 ; HILL >>>>> $OMEGA 0.083333333 ; PPV_HILL = 1/12 $SIGMA 0.000001 FIX ; EPS1 >>>>> >>>>> $SIM (1234) (5678 UNIFORM) NSUB=10 $EST METHOD=COND MAX=9990 SIG=3 >>>>> $PRED IF (ICALL.EQ.4) THEN >>>>> IF (NEWIND.LE.1) THEN >>>>> CALL RANDOM(2,R) >>>>> UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12 >>>>> HILL=THETA(1)*EXP(UNIETA) >>>>> Y=1.1**HILL/(1.1**HILL+1) >>>>> ENDIF >>>>> ELSE >>>>> >>>>> HILL=THETA(1)*EXP(ETA(1)) >>>>> Y=1.1**HILL/(1.1**HILL+1) + EPS(1) ENDIF >>>>> >>>>> REP=IREP >>>>> >>>>> $TABLE ID REP HILL UNIETA ETA(1) Y ONEHEADER NOPRINT FILE=uni.fit >>>>> >>>>> I realized after a bit more thought that my suggestion to >>>>> transform the eta value for estimation wasn't rational so please >>>>> ignore that senior moment in my earlier email on this topic. >>>>> >>>>> Nick >>>>> >>>>> >>>>> -- >>>>> >>>>> Nick Holford, Professor Clinical Pharmacology >>>>> >>>>> Dept Pharmacology & Clinical Pharmacology >>>>> >>>>> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New >>>>> Zealand >>>>> >>>>> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 >>>>> >>>>> email: [email protected] <mailto:[email protected]> >>>>> >>>>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>>>> >>> >>> -- >>> Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & >>> Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag >>> 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 >>> mobile:+64(21)46 23 53 >>> email: [email protected] >>> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford >>> > > -- > Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & > Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag > 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 > mobile:+64(21)46 23 53 > email: [email protected] > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited --- Please immediately and permanently delete.

________________________________ From: "Ribbing, Jakob" <[email protected]> To: nmusers <[email protected]> Sent: Tue, June 1, 2010 3:10:44 AM Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Dear all, Dropping in a little late in the game all I can say is this: Shame on all you great minds for reinventing your own wisdom :>) Most of the content in the current thread has already been discussed in an earlier thread: http://www.mail-archive.com/[email protected]/msg01271.html However, this old thread does contain a lot of postings and quite a few which are VERY confusing, so you may want to skip ahead to Matt’s posting here: http://www.mail-archive.com/[email protected]/msg01302.html (There are also many other postings which are very useful, but the one above captures the essence with regards to the original question in the current thread) That said I think there are always new learnings in each thread, as people tend to express themselves differently and the original question branch into several new discussion points. So I guess there are never two threads that are exactly alike, even when the usual suspects participate in both. Cheers Jakob

________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Tuesday, June 01, 2010 10:04 AM To: Ribbing, Jakob; nmusers Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Dear Jakob and all, In this thread, Mats clearly indicated Eta is assumed normal distributed. But, others have said differently. I wonder which statement is correct? ________________________________ From: "Ribbing, Jakob" <[email protected]> To: nmusers <[email protected]> Sent: Tue, June 1, 2010 3:10:44 AM Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Dear all, Dropping in a little late in the game all I can say is this: Shame on all you great minds for reinventing your own wisdom :>) Most of the content in the current thread has already been discussed in an earlier thread: http://www.mail-archive.com/[email protected]/msg01271.html However, this old thread does contain a lot of postings and quite a few which are VERY confusing, so you may want to skip ahead to Matt's posting here: http://www.mail-archive.com/[email protected]/msg01302.html (There are also many other postings which are very useful, but the one above captures the essence with regards to the original question in the current thread) That said I think there are always new learnings in each thread, as people tend to express themselves differently and the original question branch into several new discussion points. So I guess there are never two threads that are exactly alike, even when the usual suspects participate in both. Cheers Jakob

From: Mats Karlsson [mailto:[email protected]] Sent: Tuesday, June 01, 2010 4:03 PM To: 'Nick Holford'; '[email protected]' Subject: RE: [NMusers] distribution assumption of Eta in NONMEM Nick, I don’t think the design was bad at all. Two very precisely measured observations per subject with 100 subjects for determining one THETA, one OMEGA and one sigma is indeed a much more informative design than we ever get in real life. I’m not sure what you try to achieve with these simulations. The question of sensitivity to the underlying distribution and a preference for transformations that result in normally distributed ETAs (ie differences between the individual parameters and the typical parameters under the model) I think has been shown. You may find situations where it is more or less sensitive, but that does not alter the fact. You don’t provide information about estimated sigma in your example below. Was the estimate unbiased? When you compare your original uniform eta distribution with the logit-transformation, you have to look at the transformed etas. Mats Mats Karlsson, PhD Professor of Pharmacometrics Dept of Pharmaceutical Biosciences Uppsala University Box 591 751 24 Uppsala Sweden phone: +46 18 4714105 fax: +46 18 471 4003 From: Nick Holford [mailto:[email protected]] Sent: Tuesday, June 01, 2010 3:23 PM To: Mats Karlsson; [email protected] Cc: 'Marc Lavielle' Subject: Re: [NMusers] distribution assumption of Eta in NONMEM Mats, Thanks for the suggestion to try a more complex model. I agree there might be some bias in the OMEGA(1,1) estimate from uniform simulated ETA when SIGMA is estimated with 2 obs/subject. In case this was due to a rather poor design (which is not what we are trying to test) I tried your example with 10 obs/subject. Although the OMEGA(1,1) (PPV_HILL) is indeed larger than the true value the 95% parametric bootstrap confidence interval includes the true value so I would not conclude this was a significant bias. Uniform Statistic HILL PPV_HILL Obj TRUE 5 0.083333 . average 4.9583 0.093377 -16926.4 CV 0.033317 0.102836 -0.00066 0.025 4.66 0.074833 -16950.2 0.975 5.25 0.11005 -16907.7 SD 0.165194 0.009603 11.15514 N 100 I also tried using the logistic transform you suggested and got these estimates: Logistic Statistic HILL LGPAR1 LGPAR2 PPV_HILL OBJ TRUE . . . . . average 5.0926 0.58006 1.6117 1.214079 -16938.7 CV 0.049328 0.121019 0.678749 0.432531 -0.00059 0.025 4.65475 0.47075 1.15475 0.321125 -16959.9 0.975 5.45575 0.6923 2.68925 2.1435 -16920.7 SD 0.251206 0.070198 1.09394 0.525127 10.05781 N 100 As you noted the OBJ was lower on average (12.3) with the LGST model. I tried simulating from the average estimates above using these two models. The distribution for the simulated uniform UNIETA value looked reasonably flat and within -0.5 to 0.5 as expected. The ETA1 distribution simulated from the uniform model was more or less normal with most of the values between -0.5 and 0.5. However the ETA1 distribution simulated from the logistic estimation model, while also more or less normal, had most of the values lying between -2 and 2 and more than 66% outside the range -0.5 to 0.5. So although the OFV was lower with the logistic transformation this would not be a good way to simulate the original data.