AW: Simulations with/without residual error

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: 09 July 2009 05:18 To: nmusers Subject: Re: AW: [NMusers] Simulations with/without residual error Andreas, I think you know the answer to your own question! As I indicated before the use of residual error depends on the purpose of the simulation. If you want to simulate future measurements then residual error should be included.... Nick andreas lindauer wrote: > Nick, > Thank you very much for your comments. > Indeed for VPC et al. i always simulate with residual error. > I understand that when one wants to simulate the 'true' value residual > error is not needed. But what if one wants to simulate 'real' values > which will be observed in a future study. For example, you have a > PK/PD model for an anti-hypertensive drug and want to predict how many > subjects will attain a blood pressure below a pre-defined value. > Wouldn't a simulation without residual error result in an > overoptimistic prediction because in reality blood pressure is measured with > error? > On the other hand, the estimated residual error does not only reflect > measurement error but also model misspecification etc.. So, might it > be an option to simulate not with the estimated residual error but > rather with a residual error set to the imprecision of the measurement method? > Best regards, Andreas. > > > . > > -----Ursprüngliche Nachricht----- > Von: [email protected] > [mailto:[email protected]] Im Auftrag von Nick Holford > Gesendet: Mittwoch, 8. Juli 2009 15:39 > An: nmusers > Betreff: Re: [NMusers] Simulations with/without residual error > > Andreas, > > My suggestion: > > If you want to compare your simulations with actual observations then > you should include residual error in the simulation. The observations > will include noise as well as the 'true' value so in order to compare > observations with simulated observations you need the residual error. > > If you want to use the simulation to describe the 'true' value then > dont include the residual error. Residual error is assumed to have a > mean of zero around the 'true' value so there is no point in adding > this kind of noise if you are trying to predict the 'true' value. > > Your examples suggest to me that you are trying to predict the 'true' > value -- not trying to match simulations directly with measured values. > If my guess is correct then you dont need to include residual error. > > However, if you are using simulations for some kind of predictive > check (visual, numerical, statistical) that will be compared to > distribution statistics of the observations then you should include residual > error. > > Nick > > andreas lindauer wrote: > >> Dear NMUSERS, >> >> >> >> The recent discussion about simulation with a nonparametric method >> brought a general question concerning monte-carlo simulations into my >> mind. When should simulations be performed with residual error and >> when not. I am especially interested in comments regarding the >> following scenarios when the result of the simulation should be >> reported as mean or median and 90% prediction interval: >> >> 1. Simulated response at a particular time point (eg. Trough values) >> >> 2. Simulated response at a particular time point (x) relative to >> baseline response (IPRED(t=x)/IPRED(t=0) vs. DV(t=x)/DV(t=0) ) >> >> 3. Simulated time of maximal response (eg. Tmax) >> >> >> >> >> >> Thanks and best regards, Andreas. >> >> >> >> >> >> ____________________________ >> >> >> >> Andreas Lindauer >> >> >> >> Department of Clinical Pharmacy >> >> Institute of Pharmacy >> >> University of Bonn >> >> An der Immenburg 4 >> >> D-53121 Bonn >> >> >> >> phone: + 49 228 73 5781 >> >> fax: + 49 228 73 9757 >> >> >> >> > > -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Friday, July 10, 2009 12:35 AM To: nmusers Subject: Re: AW: [NMusers] Simulations with/without residual error Andreas, Thanks for your comments. I am sorry I did not explain everything that NONMEM was doing in this simple example. It does not shave my face (!) but it does recognize the covariance between the uncertainty estimates UNC_POP_CL and UNC_POP_V when the values of ETA(3) and ETA(4) are sampled -- so the covariance of 0.1 in the OMEGA block defining parameter uncertainty is not ignored. NONMEM is doing exactly the same thing you describe in R -- it is sampling from multivariate normal distributions. The code I gave was just a simple example showing the idea. Of course, you can include the full variance-covariance matrix of the estimate from a previous run including the uncertainties in THETA, OMEGA and SIGMA (and their correlations). You can also apply the uncertainties to the random effect parameters, OMEGA and SIGMA, but it may not be so simple as for THETA. I personally have no experience of this. I am sure there are others who have done it who may wish to comment. Best wishes, Nick [email protected] wrote: > > Nick, > > your example shows there is almost nothing you can not do with nonmem > (maybe except shaving your face). > On the other hand, even in the simple two parameter example you have > off-diagonal covariance terms. > In your example code the value of 0.1 in the $OMEGA block seems > ignored (covariance pop CL and pop Vol). > > There would typically also be covariances between the pop parameters > and the OMEGA and SIGMA blocks. > The latter are often small compared to other variance terms but the > proper way would be to draw from the full variance-covariance matrix. > For now it seems best to draw multivariate Normals with full > covariance matrices in some other environment like R and write the > generated population parameters to nonmem control streams. > Unless you find a way again of doing it all in nonmem. > > Best regards, > > Andreas > > PS. Specifying the variance-covariance matrix to use it with $SIM > might actually be a good candidate for the to-do list for nonmem VIII. > > > ----- > > Andreas Krause, PhD > Lead Scientist Modeling and Simulation > > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > Switzerland > [email protected] / www.actelion.com > > > > *Nick Holford <[email protected]>* > Sent by: [email protected] > > 07/09/2009 11:51 AM > > > To > nmusers <[email protected]> > cc > > Subject > Re: AW: [NMusers] Simulations with/without residual error > > > > > > > > > > Andreas K, > > It is not strictly true to say you cannot specify the parameter > uncertainties from a previous run to be included in a simulation. > > If you take the variance-covariance matrix of the estimate from a > previous run ('the uncertainty matrix') you can add it as an additional > OMEGA matrix and use it to obtain parameter samples with uncertainty. > > e.g. with a very simple example with just two parameters. This will > simulate 100 data sets and uncertainty to the THETA values for CL and V. > > $SIM (20090709) ONLYSIM SUBPROBLEMS=100 > ; estimates of THETA and OMEGA from previous run > $THETA > 1 ; POP_CL theta1 > 10 ; POP_V theta2 > $OMEGA > 0.5 ; PPV_CL eta1 > 0.5 ; PPV_V eta2 > ;variance-covariance matrix of the THETA estimates from previous run > $OMEGA BLOCK(2) > 0.2 ; UNC_POP_CL eta3 > 0.1 3 ; UNC_POP_V eta 4 > > $PK > ; get CL and V uncertainties > IF (NEWIND.EQ.0) THEN ; do this just once per subproblem > UNCCL=THETA(1)+ETA(3) > UNCV=THETA(2)+ETA(4) > ENDIF > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL > V =UNCV*EXP(ETA(2)) ; with uncertainty for V > ... > > Nick > > > > [email protected] wrote: > > Andreas, > > > > I guess you are hinting at the difference between simulation of a large > > population and simulation of a study. > > > > The latter incorporates the added uncertainty of the parameter > estimates, > > as you point out. > > You would simulate the population parameters with their > uncertainties first > > (from the "big covariance matrix" in nonmem) and then simulate the study > > with these sampled population parameters (both steps many times). > > Nonmem can only do the latter directly since you cannot specify the > > parameter uncertainties from a previous run to be included in the > > simulation. > > It is fairly straightforward though since the matrix reflects a > > multivariate Normal distribution. > > > > Andreas > > > > ----- > > > > Andreas Krause, PhD > > Lead Scientist Modeling and Simulation > > > > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > > Switzerland > > [email protected] / www.actelion.com > > > > > > > > [email protected] wrote: ----- > > > > > > To: <[email protected]> > > From: "andreas lindauer" <[email protected]> > > Sent by: [email protected] > > Date: 2009-07-09 09:42 > > Subject: AW: [NMusers] Simulations with/without residual error > > > > Nick, > > Thank you very much for your comments. > > Indeed for VPC et al. i always simulate with residual error. > > I understand that when one wants to simulate the 'true' value residual > > error > > is not needed. But what if one wants to simulate 'real' values which > will > > be > > observed in a future study. For example, you have a PK/PD model for an > > anti-hypertensive drug and want to predict how many subjects will > attain a > > blood pressure below a pre-defined value. Wouldn't a simulation without > > residual error result in an overoptimistic prediction because in reality > > blood pressure is measured with error? > > On the other hand, the estimated residual error does not only reflect > > measurement error but also model misspecification etc.. So, might it > be an > > option to simulate not with the estimated residual error but rather > with a > > residual error set to the imprecision of the measurement method? > > Best regards, Andreas. > > > > > > . > > > > -----Ursprüngliche Nachricht----- > > Von: [email protected] > [mailto:[email protected]] Im > > Auftrag von Nick Holford > > Gesendet: Mittwoch, 8. Juli 2009 15:39 > > An: nmusers > > Betreff: Re: [NMusers] Simulations with/without residual error > > > > Andreas, > > > > My suggestion: > > > > If you want to compare your simulations with actual observations then > > you should include residual error in the simulation. The observations > > will include noise as well as the 'true' value so in order to compare > > observations with simulated observations you need the residual error. > > > > If you want to use the simulation to describe the 'true' value then dont > > include the residual error. Residual error is assumed to have a mean of > > zero around the 'true' value so there is no point in adding this kind of > > noise if you are trying to predict the 'true' value. > > > > Your examples suggest to me that you are trying to predict the 'true' > > value -- not trying to match simulations directly with measured values. > > If my guess is correct then you dont need to include residual error. > > > > However, if you are using simulations for some kind of predictive check > > (visual, numerical, statistical) that will be compared to distribution > > statistics of the observations then you should include residual error. > > > > Nick > > > > andreas lindauer wrote: > > > >> Dear NMUSERS, > >> > >> > >> > >> The recent discussion about simulation with a nonparametric method > >> brought a general question concerning monte-carlo simulations into my > >> mind. When should simulations be performed with residual error and > >> when not. I am especially interested in comments regarding the > >> following scenarios when the result of the simulation should be > >> reported as mean or median and 90% prediction interval: > >> > >> 1. Simulated response at a particular time point (eg. Trough values) > >> > >> 2. Simulated response at a particular time point (x) relative to > >> baseline response (IPRED(t=x)/IPRED(t=0) vs. DV(t=x)/DV(t=0) ) > >> > >> 3. Simulated time of maximal response (eg. Tmax) > >> > >> > >> > >> > >> > >> Thanks and best regards, Andreas. > >> > >> > >> > >> > >> > >> ____________________________ > >> > >> > >> > >> Andreas Lindauer > >> > >> > >> > >> Department of Clinical Pharmacy > >> > >> Institute of Pharmacy > >> > >> University of Bonn > >> > >> An der Immenburg 4 > >> > >> D-53121 Bonn > >> > >> > >> > >> phone: + 49 228 73 5781 > >> > >> fax: + 49 228 73 9757 > >> > >> > >> > >> > > > > -- > > Nick Holford, Professor Clinical Pharmacology > > Dept Pharmacology & Clinical Pharmacology > > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New > > Zealand > > [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 > > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) > > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > > > > > > > > The information of this email and in any file transmitted with it is > strictly confidential and may be legally privileged. > > It is intended solely for the addressee. If you are not the intended > recipient, any copying, distribution or any other use of this email is > prohibited and may be unlawful. In such case, you should please notify > the sender immediately and destroy this email. > > The content of this email is not legally binding unless confirmed by > letter. > > Any views expressed in this message are those of the individual > sender, except where the message states otherwise and the sender is > authorised to state them to be the views of the sender's company. For > further information about Actelion please see our website at > http://www.actelion.com > > > > > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New > Zealand > [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > > > > The information of this email and in any file transmitted with it is strictly > confidential and may be legally privileged. > It is intended solely for the addressee. If you are not the intended > recipient, any copying, distribution or any other use of this email is > prohibited and may be unlawful. In such case, you should please notify the > sender immediately and destroy this email. > The content of this email is not legally binding unless confirmed by letter. > Any views expressed in this message are those of the individual sender, > except where the message states otherwise and the sender is authorised to > state them to be the views of the sender's company. For further information > about Actelion please see our website at http://www.actelion.com > -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

________________________________ From: Nick Holford <[email protected]> To: nmusers <[email protected]> Sent: Friday, July 10, 2009 3:34:50 AM Subject: Re: AW: [NMusers] Simulations with/without residual error Andreas, Thanks for your comments. I am sorry I did not explain everything that NONMEM was doing in this simple example. It does not shave my face (!) but it does recognize the covariance between the uncertainty estimates UNC_POP_CL and UNC_POP_V when the values of ETA(3) and ETA(4) are sampled -- so the covariance of 0.1 in the OMEGA block defining parameter uncertainty is not ignored. NONMEM is doing exactly the same thing you describe in R -- it is sampling from multivariate normal distributions. The code I gave was just a simple example showing the idea. Of course, you can include the full variance-covariance matrix of the estimate from a previous run including the uncertainties in THETA, OMEGA and SIGMA (and their correlations). You can also apply the uncertainties to the random effect parameters, OMEGA and SIGMA, but it may not be so simple as for THETA. I personally have no experience of this. I am sure there are others who have done it who may wish to comment. Best wishes, Nick [email protected] wrote: > > Nick, > > your example shows there is almost nothing you can not do with nonmem (maybe > except shaving your face). > On the other hand, even in the simple two parameter example you have > off-diagonal covariance terms. > In your example code the value of 0.1 in the $OMEGA block seems ignored > (covariance pop CL and pop Vol). > > There would typically also be covariances between the pop parameters and the > OMEGA and SIGMA blocks. > The latter are often small compared to other variance terms but the proper > way would be to draw from the full variance-covariance matrix. > For now it seems best to draw multivariate Normals with full covariance > matrices in some other environment like R and write the generated population > parameters to nonmem control streams. > Unless you find a way again of doing it all in nonmem. > > Best regards, > > Andreas > > PS. Specifying the variance-covariance matrix to use it with $SIM might > actually be a good candidate for the to-do list for nonmem VIII. > > > ----- > > Andreas Krause, PhD > Lead Scientist Modeling and Simulation > > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > Switzerland > [email protected] / www.actelion.com > > > > *Nick Holford <[email protected]>* > Sent by: [email protected] > > 07/09/2009 11:51 AM > > > To > nmusers <[email protected]> > cc > > Subject > Re: AW: [NMusers] Simulations with/without residual error > > > > > > > > > > Andreas K, > > It is not strictly true to say you cannot specify the parameter > uncertainties from a previous run to be included in a simulation. > > If you take the variance-covariance matrix of the estimate from a > previous run ('the uncertainty matrix') you can add it as an additional > OMEGA matrix and use it to obtain parameter samples with uncertainty. > > e.g. with a very simple example with just two parameters. This will > simulate 100 data sets and uncertainty to the THETA values for CL and V. > > $SIM (20090709) ONLYSIM SUBPROBLEMS=100 > ; estimates of THETA and OMEGA from previous run > $THETA > 1 ; POP_CL theta1 > 10 ; POP_V theta2 > $OMEGA > 0.5 ; PPV_CL eta1 > 0.5 ; PPV_V eta2 > ;variance-covariance matrix of the THETA estimates from previous run > $OMEGA BLOCK(2) > 0.2 ; UNC_POP_CL eta3 > 0.1 3 ; UNC_POP_V eta 4 > > $PK > ; get CL and V uncertainties > IF (NEWIND.EQ.0) THEN ; do this just once per subproblem > UNCCL=THETA(1)+ETA(3) > UNCV=THETA(2)+ETA(4) > ENDIF > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL > V =UNCV*EXP(ETA(2)) ; with uncertainty for V > ... > > Nick > > > > [email protected] wrote: > > Andreas, > > > > I guess you are hinting at the difference between simulation of a large > > population and simulation of a study. > > > > The latter incorporates the added uncertainty of the parameter estimates, > > as you point out. > > You would simulate the population parameters with their uncertainties first > > (from the "big covariance matrix" in nonmem) and then simulate the study > > with these sampled population parameters (both steps many times). > > Nonmem can only do the latter directly since you cannot specify the > > parameter uncertainties from a previous run to be included in the > > simulation. > > It is fairly straightforward though since the matrix reflects a > > multivariate Normal distribution. > > > > Andreas > > > > ----- > > > > Andreas Krause, PhD > > Lead Scientist Modeling and Simulation > > > > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > > Switzerland > > [email protected]/ www.actelion.com > > > > > > > > [email protected] wrote: ----- > > > > > > To: <[email protected]> > > From: "andreas lindauer" <[email protected]> > > Sent by: [email protected] > > Date: 2009-07-09 09:42 > > Subject: AW: [NMusers] Simulations with/without residual error > > > > Nick, > > Thank you very much for your comments. > > Indeed for VPC et al. i always simulate with residual error. > > I understand that when one wants to simulate the 'true' value residual > > error > > is not needed. But what if one wants to simulate 'real' values which will > > be > > observed in a future study. For example, you have a PK/PD model for an > > anti-hypertensive drug and want to predict how many subjects will attain a > > blood pressure below a pre-defined value. Wouldn't a simulation without > > residual error result in an overoptimistic prediction because in reality > > blood pressure is measured with error? > > On the other hand, the estimated residual error does not only reflect > > measurement error but also model misspecification etc.. So, might it be an > > option to simulate not with the estimated residual error but rather with a > > residual error set to the imprecision of the measurement method? > > Best regards, Andreas. > > > > > > . > > > > -----Ursprüngliche Nachricht----- > > Von: [email protected] [mailto:[email protected]] Im > > Auftrag von Nick Holford > > Gesendet: Mittwoch, 8. Juli 2009 15:39 > > An: nmusers > > Betreff: Re: [NMusers] Simulations with/without residual error > > > > Andreas, > > > > My suggestion: > > > > If you want to compare your simulations with actual observations then > > you should include residual error in the simulation. The observations > > will include noise as well as the 'true' value so in order to compare > > observations with simulated observations you need the residual error. > > > > If you want to use the simulation to describe the 'true' value then dont > > include the residual error. Residual error is assumed to have a mean of > > zero around the 'true' value so there is no point in adding this kind of > > noise if you are trying to predict the 'true' value. > > > > Your examples suggest to me that you are trying to predict the 'true' > > value -- not trying to match simulations directly with measured values. > > If my guess is correct then you dont need to include residual error. > > > > However, if you are using simulations for some kind of predictive check > > (visual, numerical, statistical) that will be compared to distribution > > statistics of the observations then you should include residual error. > > > > Nick > > > > andreas lindauer wrote: > > >> Dear NMUSERS, > >> > >> > >> > >> The recent discussion about simulation with a nonparametric method > >> brought a general question concerning monte-carlo simulations into my > >> mind. When should simulations be performed with residual error and > >> when not. I am especially interested in comments regarding the > >> following scenarios when the result of the simulation should be > >> reported as mean or median and 90% prediction interval: > >> > >> 1. Simulated response at a particular time point (eg. Trough values) > >> > >> 2. Simulated response at a particular time point (x) relative to > >> baseline response (IPRED(t=x)/IPRED(t=0) vs. DV(t=x)/DV(t=0) ) > >> > >> 3. Simulated time of maximal response (eg. Tmax) > >> > >> > >> > >> > >> > >> Thanks and best regards, Andreas. > >> > >> > >> > >> > >> > >> ____________________________ > >> > >> > >> > >> Andreas Lindauer > >> > >> > >> > >> Department of Clinical Pharmacy > >> > >> Institute of Pharmacy > >> > >> University of Bonn > >> > >> An der Immenburg 4 > >> > >> D-53121 Bonn > >> > >> > >> > >> phone: + 49 228 73 5781 > >> > >> fax: + 49 228 73 9757 > >> > >> > >> > >> > > > -- > > Nick Holford, Professor Clinical Pharmacology > > Dept Pharmacology & Clinical Pharmacology > > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New > > Zealand > > [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 > > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) > > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > > > > > > > > The information of this email and in any file transmitted with it is > > strictly confidential and may be legally privileged. > > It is intended solely for the addressee. If you are not the intended > > recipient, any copying, distribution or any other use of this email is > > prohibited and may be unlawful. In such case, you should please notify the > > sender immediately and destroy this email. > > The content of this email is not legally binding unless confirmed by letter. > > Any views expressed in this message are those of the individual sender, > > except where the message states otherwise and the sender is authorised to > > state them to be the views of the sender's company. For further information > > about Actelion please see our website at http://www.actelion.com > > > > > -- Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand > [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford > > > > The information of this email and in any file transmitted with it is strictly > confidential and may be legally privileged. > It is intended solely for the addressee. If you are not the intended > recipient, any copying, distribution or any other use of this email is > prohibited and may be unlawful. In such case, you should please notify the > sender immediately and destroy this email. > The content of this email is not legally binding unless confirmed by letter. > Any views expressed in this message are those of the individual sender, > except where the message states otherwise and the sender is authorised to > state them to be the views of the sender's company. For further information > about Actelion please see our website at http://www.actelion.com -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +33 64 271-6369 (Apr 6-Jul 20 2009) http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

-----Original Message----- From: [email protected] [mailto:[email protected]]on Behalf Of Gastonguay, Marc Sent: 16 July 2009 18:59 To: nmusers Subject: Re: AW: [NMusers] Simulations with/without residual error (Apologies for the delayed posting.. this apparently didn't make it to nmusers on the initial attempt). Dear Nick, Andreas, Andreas and nmusers, Here are a couple of additional methods for including uncertainty in parameters at the inter-trial (or inter-replicate) level, when simulating with NONMEM: 1. You can take advantage the PRIOR subroutine in NONMEM VI (and VII - although I haven't tried it yet) simulations, to generate random variates from a Multi-Variate Normal distribution for THETA and an Inverse Wishart distribution for OMEGA. This works fine if your prior uncertainty distributions are adequately described by these distributions. Of course the MVN assumption is consistent with the var-covar matrix of the estimates in NONMEM, but you'll have to translate the uncertainty in OMEGA into the required parameters of an Inv. Wishart (e.g. mode and degrees of freedom). This method does not directly allow for prior uncertainty on SIGMA. 2. If you'd like to simulate from other distributions, or pull-in uncertainty in parameter estimates from other sources, such as the resulting parameter estimates from bootstrap replicates or MCMC Bayesian posterior distributions, you'll need to use an external tool with NONMEM. As Andreas points out, R is a useful choice. Leonid Gibiasnky and I had developed a toolkit of R functions called NMSUDS to facilitate these types of simulations in NONMEM. These functions have been extended and are now part of the broader MIfuns package ( http://cran.r-project.org/). There's another important issue to consider... Be careful that the specification of the prior uncertainty distribution is consistent with reality for the parameters in your model. This point has been discussed by Pascal Girard and others in past nmusers threads. For example, a MVN uncertainty distribution for THETA is not realistic for PK parameters and is never realistic for OMEGA and SIGMA, in that MVN allows for simulation of negative values. To work-around this problem for THETA, you could choose to log-transform typical values of PK parameters to constrain resulting replicates within a physiologically realistic range. For example: Instead of: CL = THETA(1)*(WT/70)**THETA(2)*EXP(ETA(1)) Parameterize as: LNCL = THETA(1)+THETA(2)*(WT/70)+ETA(1) CL = EXP(LNCL) This sort of transformation is a useful thing to do for NONMEM simulation and estimation in general, because it creates a parameter uncertainty distribution that is consistent (for THETA) with the MVN assumption implicit in Maximum Likelihood methods for continuous data. This means that confidence intervals (for THETA) from NONMEM's asymptotic standard errors ($COV) should be more realistic. You may also find improved stability in estimation runs. Best regards, Marc Marc R. Gastonguay, Ph.D. < [email protected] > President & CEO, Metrum Research Group LLC < metrumrg.com > Scientific Director, Metrum Institute < metruminstitute.org > 2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct: +1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014

On Jul 17, 2009, at 4:13 AM, Grevel, Joachim wrote: > Dear Marc, > > I am sorry, but I am missing your boat. You wrote: > > For example: > > Instead of: > CL = THETA(1)*(WT/70)**THETA(2)*EXP(ETA(1)) > > Parameterize as: > LNCL = THETA(1)+THETA(2)*(WT/70)+ETA(1) > CL = EXP(LNCL) > > This sort of transformation is a useful thing to do for NONMEM simulation and estimation in general, because it creates a parameter uncertainty distribution that is consistent (for THETA) with the MVN assumption implicit in Maximum Likelihood methods for continuous data. This means that confidence intervals (for THETA) from NONMEM's asymptotic standard errors ($COV) should be more realistic. You may also find improved stability in estimation runs. > > Best regards, > Marc > > How can your first line of your code ever result in negative CL. I have adopted the log-transformation of data before estimation (thanks to Matts for promoting this!), but I cannot see the reason why to log-transform parameters before simulation when I use proportional error terms. > > Thanks, > > Joachim > _________________________________ > AstraZeneca R&D Charnwood > Clin. Pharmacology and DMP > Bakewell Road > Loughborough, LE11 5RH > Tel: +44 1509 644035 > [email protected] > > AstraZeneca UK Limited is a company incorporated in England and Wales with registered number: 03674842 and a registered office at 15 Stanhope Gate, London W1K 1LN. Confidentiality Notice: This message is private and may contain confidential, proprietary and legally privileged information. If you have received this message in error, please notify us and remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorised use or disclosure of the contents of this message is not permitted and may be unlawful. Disclaimer: Email messages may be subject to delays, interception, non-delivery and unauthorised alterations. Therefore, information expressed in this message is not given or endorsed by AstraZeneca UK Limited unless otherwise notified by an authorised representative independent of this message. No contractual relationship is created by this message by any person unless specifically indicated by agreement in writing other than email. Monitoring: AstraZeneca UK Limited may monitor email traffic data and content for the purposes of the prevention and detection of crime, ensuring the security of our computer systems and checking compliance with our Code of Conduct and policies. > > -----Original Message----- > > From: [email protected] [ mailto: [email protected] ]On Behalf Of Gastonguay, Marc > > Sent: 16 July 2009 18:59 > To: nmusers > Subject: Re: AW: [NMusers] Simulations with/without residual error > > (Apologies for the delayed posting.. this apparently didn't make it to nmusers on the initial attempt). > > Dear Nick, Andreas, Andreas and nmusers, > > Here are a couple of additional methods for including uncertainty in parameters at the inter-trial (or inter-replicate) level, when simulating with NONMEM: > > 1. You can take advantage the PRIOR subroutine in NONMEM VI (and VII - although I haven't tried it yet) simulations, to generate random variates from a Multi-Variate Normal distribution for THETA and an Inverse Wishart distribution for OMEGA. This works fine if your prior uncertainty distributions are adequately described by these distributions. Of course the MVN assumption is consistent with the var-covar matrix of the estimates in NONMEM, but you'll have to translate the uncertainty in OMEGA into the required parameters of an Inv. Wishart (e.g. mode and degrees of freedom). This method does not directly allow for prior uncertainty on SIGMA. > > 2. If you'd like to simulate from other distributions, or pull-in uncertainty in parameter estimates from other sources, such as the resulting parameter estimates from bootstrap replicates or MCMC Bayesian posterior distributions, you'll need to use an external tool with NONMEM. As Andreas points out, R is a useful choice. Leonid Gibiasnky and I had developed a toolkit of R functions called NMSUDS to facilitate these types of simulations in NONMEM. These functions have been extended and are now part of the broader MIfuns package ( http://cran.r-project.org/ ). > > There's another important issue to consider... Be careful that the specification of the prior uncertainty distribution is consistent with reality for the parameters in your model. This point has been discussed by Pascal Girard and others in past nmusers threads. For example, a MVN uncertainty distribution for THETA is not realistic for PK parameters and is never realistic for OMEGA and SIGMA, in that MVN allows for simulation of negative values. To work-around this problem for THETA, you could choose to log-transform typical values of PK parameters to constrain resulting replicates within a physiologically realistic range. > > For example: > > Instead of: > CL = THETA(1)*(WT/70)**THETA(2)*EXP(ETA(1)) > > Parameterize as: > LNCL = THETA(1)+THETA(2)*(WT/70)+ETA(1) > CL = EXP(LNCL) > > This sort of transformation is a useful thing to do for NONMEM simulation and estimation in general, because it creates a parameter uncertainty distribution that is consistent (for THETA) with the MVN assumption implicit in Maximum Likelihood methods for continuous data. This means that confidence intervals (for THETA) from NONMEM's asymptotic standard errors ($COV) should be more realistic. You may also find improved stability in estimation runs. > > Best regards, > Marc > > Marc R. Gastonguay, Ph.D. < [email protected] > > President & CEO, Metrum Research Group LLC < metrumrg.com > > Scientific Director, Metrum Institute < metruminstitute.org > > > 2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct: +1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Sunday, July 26, 2009 10:42 PM To: nmusers Subject: Re: AW: [NMusers] Simulations with/without residual error Mats, Thanks for extending the code fragment I proposed earlier to add some extra useful features. I don't know what was incorrect about the code I suggested but resampling the uncertainty ETA values is a pragmatic way to avoid PREDPP errors caused by randomly generated non-positive CL or V values. The code below produces the same results as your code but it shows how to check on reasonable values for CL and V that do not depend on the value of CL or V defined by THETA or other things such as the effect of covariates on the group parameter value. I note that your example using simulation and estimation does not use the same residual error model for simulation and estimation. The simulation residual error is exp(EPS(1)) but METHOD=CONDITIONAL INTERACTION would approximate this by (1+EPS(1)). You may remember the commentary on this point that Stuart Beal wrote in 2002 :-) Best wishes, Nick Beal SL. Commentary on Significance Levels for Covariate Effects in NONMEM. Journal of Pharmacokinetics & Pharmacodynamics. 2002;29(4):403-10. $SIM (20090726 NEW) ONLYSIM SUBPROBLEMS=100 ; estimates of THETA and OMEGA from previous run $THETA 1 ; POP_CL theta1 10 ; POP_V theta2 $OMEGA 0.5 ; PPV_CL eta1 0.5 ; PPV_V eta2 ;variance-covariance matrix of the THETA estimates from previous run $OMEGA BLOCK(2) 0.2 ; UNC_POP_CL eta3 0.1 3 ; UNC_POP_V eta 4 $SIGMA .01 $SUB ADVAN1 $PK ; get CL and V uncertainties IF (ICALL.EQ.4.AND.NEWIND.EQ.0) THEN ; do this just once per subproblem UNCCL=THETA(1)+ETA(3) UNCV=THETA(2)+ETA(4) DO WHILE (UNCCL.LE.0.OR.UNCV.LE.0) ;resample if values unreasonable CALL SIMETA(ETA) UNCCL=THETA(1)+ETA(3) UNCV=THETA(2)+ETA(4) END DO ENDIF CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL V =UNCV*EXP(ETA(2)) ; with uncertainty for V K=CL/V $ERROR Y =F*EXP(EPS(1)) Mats Karlsson wrote: > > Dear Nick, Marc and all, > > An even later addition to this thread. Nick pointed out the > possibility to do simulations in NONMEM that incorporated error in > simulation parameters. The code provided gave a good hint at how to do > that but would work incorrectly unless some additional NONMEM switches > were flicked. Also, as pointed out by Marc and others, one might like > to truncate the parameter distribution for simulation (avoid negative > CL etc). Also it may be convenient to do simulation followed by > re-estimation in a single run. For appropriate post-processing this > requires that simulation parameters are output somewhere. A model > file, extending Nick's example, that does these things are provided > below. > > $PROB > > $DATA data1 > > $INPUT ID DV TIME TVCL TVV AMT > > ;data1= > > ; 1 0 0 0 0 100 > > ; 1 0 1 0 0 0 > > ; 1 0 21 0 0 0 > > ; 2 0 0 0 0 100 > > ; etc > > $SIM (20090726 NEW) SUBPROBLEMS=100 > > $THETA ; estimates of THETA and OMEGA from previous run > > (0,1) ; POP_CL theta1 > > (0,10) ; POP_V theta2 > > $OMEGA > > 0.5 ; PPV_CL eta1 > > 0.5 ; PPV_V eta2 > > ;variance-covariance matrix of the THETA estimates from previous run > > $OMEGA BLOCK(2) > > 0.2 ; UNC_POP_CL eta3 > > 0.1 3 FIX; UNC_POP_V eta 4 > > $SIGMA .01 > > $SUB ADVAN1 > > $PK > > CL=THETA(1)*EXP(ETA(1)) ; with uncertainty for CL > > V =THETA(2)*EXP(ETA(2)) ; with uncertainty for V > > K=CL/V > > IF (ICALL.EQ.4.AND.NEWIND.EQ.0) THEN ; do this just once per subproblem > > DO WHILE (ETA(3).LT.-.99.OR.ETA(4).LT.-9.9) CALL SIMETA(ETA) ;resample > if "bad" values > > END DO > > TVCL=THETA(1)+ETA(3) > > TVV =THETA(2)+ETA(4) > > ENDIF > > IF (ICALL.EQ.4) THEN ; do this for all simulated values > > CL=TVCL*EXP(ETA(1)) ; with uncertainty for CL > > V =TVV*EXP(ETA(2)) ; with uncertainty for V > > ENDIF > > $ERROR > > Y =F*EXP(EPS(1)) > > $INFN > > IF (NEWIND.EQ.0) THEN > > WRITE (52,*) THETA ; Output estimated THETAs to file 52 > > WRITE (53,*) TVCL, TVV ; Output simulated THETAs to file 53 > > ENDIF > > $EST MAX=9999 METH=1 INTER > > Mats Karlsson, PhD > > Professor of Pharmacometrics > > Dept of Pharmaceutical Biosciences > > Uppsala University > > Box 591 > > 751 24 Uppsala Sweden > > phone: +46 18 4714105 > > fax: +46 18 471 4003 > Nick Holford wrote: > $SIM (20090709) ONLYSIM SUBPROBLEMS=100 > ; estimates of THETA and OMEGA from previous run > $THETA > 1 ; POP_CL theta1 > 10 ; POP_V theta2 > $OMEGA > 0.5 ; PPV_CL eta1 > 0.5 ; PPV_V eta2 > ;variance-covariance matrix of the THETA estimates from previous run > $OMEGA BLOCK(2) > 0.2 ; UNC_POP_CL eta3 > 0.1 3 ; UNC_POP_V eta 4 > > $PK > ; get CL and V uncertainties > IF (NEWIND.EQ.0) THEN ; do this just once per subproblem > UNCCL=THETA(1)+ETA(3) > UNCV=THETA(2)+ETA(4) > ENDIF > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL > V =UNCV*EXP(ETA(2)) ; with uncertainty for V > ... > -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +64 21 46 23 53 http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

> -----Original Message----- > From: [email protected] [mailto:[email protected]] On > Behalf Of Nick Holford > Sent: Sunday, July 26, 2009 10:42 PM > To: nmusers > Subject: Re: AW: [NMusers] Simulations with/without residual error > > Mats, > > Thanks for extending the code fragment I proposed earlier to add some extra useful features. > > I don't know what was incorrect about the code I suggested but resampling the uncertainty ETA values is a pragmatic way to avoid PREDPP errors caused by randomly generated non-positive CL or V values. The code below produces the same results as your code but it shows how to check on reasonable values for CL and V that do not depend on the value of CL or V defined by THETA or other things such as the effect of covariates on the group parameter value. > > I note that your example using simulation and estimation does not use the same residual error model for simulation and estimation. The simulation residual error is exp(EPS(1)) but METHOD=CONDITIONAL INTERACTION would approximate this by (1+EPS(1)). You may remember the commentary on this point that Stuart Beal wrote in 2002 :-) > > Best wishes, > > Nick > > Beal SL. Commentary on Significance Levels for Covariate Effects in NONMEM. Journal of Pharmacokinetics & Pharmacodynamics. 2002;29(4):403-10. > > $SIM (20090726 NEW) ONLYSIM SUBPROBLEMS=100 > ; estimates of THETA and OMEGA from previous run > $THETA > 1 ; POP_CL theta1 > 10 ; POP_V theta2 > $OMEGA > 0.5 ; PPV_CL eta1 > 0.5 ; PPV_V eta2 > ;variance-covariance matrix of the THETA estimates from previous run > $OMEGA BLOCK(2) > 0.2 ; UNC_POP_CL eta3 > 0.1 3 ; UNC_POP_V eta 4 > $SIGMA .01 > > $SUB ADVAN1 > > $PK > ; get CL and V uncertainties > IF (ICALL.EQ.4.AND.NEWIND.EQ.0) THEN ; do this just once per subproblem > UNCCL=THETA(1)+ETA(3) > UNCV=THETA(2)+ETA(4) > DO WHILE (UNCCL.LE.0.OR.UNCV.LE.0) ;resample if values unreasonable > CALL SIMETA(ETA) > UNCCL=THETA(1)+ETA(3) > UNCV=THETA(2)+ETA(4) > END DO > ENDIF > > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL > V =UNCV*EXP(ETA(2)) ; with uncertainty for V > K=CL/V > > $ERROR > Y =F*EXP(EPS(1)) > > Mats Karlsson wrote: > > > Dear Nick, Marc and all, > > > > An even later addition to this thread. Nick pointed out the possibility to do simulations in NONMEM that incorporated error in simulation parameters. The code provided gave a good hint at how to do that but would work incorrectly unless some additional NONMEM switches were flicked. Also, as pointed out by Marc and others, one might like to truncate the parameter distribution for simulation (avoid negative CL etc). Also it may be convenient to do simulation followed by re-estimation in a single run. For appropriate post-processing this requires that simulation parameters are output somewhere. A model file, extending Nick's example, that does these things are provided below. > > > > $PROB > > > > $DATA data1 > > > > $INPUT ID DV TIME TVCL TVV AMT > > > > ;data1= > > > > ; 1 0 0 0 0 100 > > > > ; 1 0 1 0 0 0 > > > > ; 1 0 21 0 0 0 > > > > ; 2 0 0 0 0 100 > > > > ; etc > > > > $SIM (20090726 NEW) SUBPROBLEMS=100 > > > > $THETA ; estimates of THETA and OMEGA from previous run > > > > (0,1) ; POP_CL theta1 > > > > (0,10) ; POP_V theta2 > > > > $OMEGA > > > > 0.5 ; PPV_CL eta1 > > > > 0.5 ; PPV_V eta2 > > > > ;variance-covariance matrix of the THETA estimates from previous run > > > > $OMEGA BLOCK(2) > > > > 0.2 ; UNC_POP_CL eta3 > > > > 0.1 3 FIX; UNC_POP_V eta 4 > > > > $SIGMA .01 > > > > $SUB ADVAN1 > > > > $PK > > > > CL=THETA(1)*EXP(ETA(1)) ; with uncertainty for CL > > > > V =THETA(2)*EXP(ETA(2)) ; with uncertainty for V > > > > K=CL/V > > > > IF (ICALL.EQ.4.AND.NEWIND.EQ.0) THEN ; do this just once per subproblem > > > > DO WHILE (ETA(3).LT.-.99.OR.ETA(4).LT.-9.9) CALL SIMETA(ETA) ;resample if "bad" values > > > > END DO > > > > TVCL=THETA(1)+ETA(3) > > > > TVV =THETA(2)+ETA(4) > > > > ENDIF > > > > IF (ICALL.EQ.4) THEN ; do this for all simulated values > > > > CL=TVCL*EXP(ETA(1)) ; with uncertainty for CL > > > > V =TVV*EXP(ETA(2)) ; with uncertainty for V > > > > ENDIF > > > > $ERROR > > > > Y =F*EXP(EPS(1)) > > > > $INFN > > > > IF (NEWIND.EQ.0) THEN > > > > WRITE (52,*) THETA ; Output estimated THETAs to file 52 > > > > WRITE (53,*) TVCL, TVV ; Output simulated THETAs to file 53 > > > > ENDIF > > > > $EST MAX=9999 METH=1 INTER > > > > Mats Karlsson, PhD > > > > Professor of Pharmacometrics > > > > Dept of Pharmaceutical Biosciences > > > > Uppsala University > > > > Box 591 > > > > 751 24 Uppsala Sweden > > > > phone: +46 18 4714105 > > > > fax: +46 18 471 4003 > > Nick Holford wrote: > > > $SIM (20090709) ONLYSIM SUBPROBLEMS=100 > > ; estimates of THETA and OMEGA from previous run > > $THETA > > 1 ; POP_CL theta1 > > 10 ; POP_V theta2 > > $OMEGA > > 0.5 ; PPV_CL eta1 > > 0.5 ; PPV_V eta2 > > ;variance-covariance matrix of the THETA estimates from previous run > > $OMEGA BLOCK(2) > > 0.2 ; UNC_POP_CL eta3 > > 0.1 3 ; UNC_POP_V eta 4 > > > > $PK > > ; get CL and V uncertainties > > IF (NEWIND.EQ.0) THEN ; do this just once per subproblem > > UNCCL=THETA(1)+ETA(3) > > UNCV=THETA(2)+ETA(4) > > ENDIF > > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL > > V =UNCV*EXP(ETA(2)) ; with uncertainty for V > > ... -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand [email protected] tel:+64(9)923-6730 fax:+64(9)373-7090 mobile: +64 21 46 23 53 http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford