Dear NMusers,
I wonder how does $Simulation work with $Estimation in NONMEM exactly?
The manual says, NONMEM will simulate DVs and replace the original DVs
based on the parameter initial values. But the initial values are not final
estimates. Does the simulation based on initial values make sense? Do
$Estimation and $Simulation run alternatively? The $Simulation generates a
set of data based on which the parameters are estimated? And then the
predictions are made on the estimated parameters? A little confused.
Appreciate any comment.
Jun
How $simulation work with $estimation
Dear Jun,
When $SIM and $EST on the same model file, the simulation will be run first,
based on initial values, and then, afterwards, the estimation of the
simulated data will be done, that's it, no further re-simulation based on
the estimated parameters will follow.
So what you should do is first to analyze your observed data in an
estimation model file, and then report your estimated parameters as initial
values to run your simulation-estimation study; this allows you to compare
estimates of real data to estimates of simulated data, so to check
simulation properties of your model.
I hope this helps,
Elodie
Elodie Plan, PharmD, MSc,
PhDstudent
*****************************************
Div. of Pharmacokinetics and Drug Therapy,
Department of Pharmaceutical Biosciences,
Faculty of Pharmacy, Uppsala University
PO - Box 591 - 751 24 Uppsala - SWEDEN
Office +46 18 4714385 - Fax +46 18 4714003
------------------------------------------------------------
Quoted reply history
From: owner-nmusers
Behalf Of Jun Shen
Sent: Wednesday, October 15, 2008 1:01 AM
To: nmusers
Subject: [NMusers] How $simulation work with $estimation
Dear NMusers,
I wonder how does $Simulation work with $Estimation in NONMEM exactly?
The manual says, NONMEM will simulate DVs and replace the original DVs
based on the parameter initial values. But the initial values are not final
estimates. Does the simulation based on initial values make sense? Do
$Estimation and $Simulation run alternatively? The $Simulation generates a
set of data based on which the parameters are estimated? And then the
predictions are made on the estimated parameters? A little confused.
Appreciate any comment.
Jun
Jun,
In addition to Elodie's clear explanation of the basic process of using
$SIM and $EST in the same control stream you may wish to know that you
don't have to use the same parameters for simulation as those used for
estimation. The ICALL variable has a value of 4 when NONMEM is
simulating so you can do this:
$THETA
1 FIX ; sim_CL
10 ; est_CL
$PK
IF (ICALL.EQ.4) THEN ; for simulation
CL=THETA(1)
ELSE ; for estimation
CL=THETA(2)
ENDIF
Nick
Elodie Plan wrote:
>
> Dear Jun,
>
> When $SIM and $EST on the same model file, the simulation will be run
> first, based on initial values, and then, afterwards, the estimation
> of the simulated data will be done, thats it, no further
> re-simulation based on the estimated parameters will follow.
>
> So what you should do is first to analyze your observed data in an
> estimation model file, and then report your estimated parameters as
> initial values to run your simulation-estimation study; this allows
> you to compare estimates of real data to estimates of simulated data,
> so to check simulation properties of your model.
>
> I hope this helps,
>
> Elodie
>
> / Elodie Plan, PharmD, MSc, ///
>
> / PhDstudent// ///
>
> /*****************************************///
>
> /Div. of Pharmacokinetics and Drug Therapy,
> Department of Pharmaceutical Biosciences,
> Faculty of Pharmacy, //Uppsala University/
>
> / PO - Box 591 - 751 24 //Uppsala - SWEDEN/
>
> /Office +46 18 4714385 - Fax +46 18 4714003
> ------------------------------------------------------------///
>
> *From:* owner-nmusers
> [mailto:owner-nmusers
> *Sent:* Wednesday, October 15, 2008 1:01 AM
> *To:* nmusers
> *Subject:* [NMusers] How $simulation work with $estimation
>
> Dear NMusers,
>
> I wonder how does $Simulation work with $Estimation in NONMEM exactly?
> The manual says, NONMEM will simulate DVs and replace the original DVs
> based on the parameter initial values. But the initial values are not
> final
> estimates. Does the simulation based on initial values make sense? Do
> $Estimation and $Simulation run alternatively? The $Simulation generates a
> set of data based on which the parameters are estimated? And then the
> predictions are made on the estimated parameters? A little confused.
>
> Appreciate any comment.
>
> Jun
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Dear Jun,
When $SIM and $EST on the same model file, the simulation will be run first,
based on initial values, and then, afterwards, the estimation of the
simulated data will be done, that's it, no further re-simulation based on
the estimated parameters will follow.
So what you should do is first to analyze your observed data in an
estimation model file, and then report your estimated parameters as initial
values to run your simulation-estimation study; this allows you to compare
estimates of real data to estimates of simulated data, so to check
simulation properties of your model.
I hope this helps,
Elodie
Elodie Plan, PharmD, MSc,
PhDstudent
*****************************************
Div. of Pharmacokinetics and Drug Therapy,
Department of Pharmaceutical Biosciences,
Faculty of Pharmacy, Uppsala University
PO - Box 591 - 751 24 Uppsala - SWEDEN
Office +46 18 4714385 - Fax +46 18 4714003
------------------------------------------------------------
Quoted reply history
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Jun Shen
Sent: Wednesday, October 15, 2008 1:01 AM
To: [email protected]
Subject: [NMusers] How $simulation work with $estimation
Dear NMusers,
I wonder how does $Simulation work with $Estimation in NONMEM exactly?
The manual says, NONMEM will simulate DVs and replace the original DVs
based on the parameter initial values. But the initial values are not final
estimates. Does the simulation based on initial values make sense? Do
$Estimation and $Simulation run alternatively? The $Simulation generates a
set of data based on which the parameters are estimated? And then the
predictions are made on the estimated parameters? A little confused.
Appreciate any comment.
Jun
Jun,
In addition to Elodie's clear explanation of the basic process of using $SIM and $EST in the same control stream you may wish to know that you don't have to use the same parameters for simulation as those used for estimation. The ICALL variable has a value of 4 when NONMEM is simulating so you can do this:
$THETA
1 FIX ; sim_CL
10 ; est_CL
$PK
IF (ICALL.EQ.4) THEN ; for simulation
CL=THETA(1)
ELSE ; for estimation
CL=THETA(2)
ENDIF
Nick
Elodie Plan wrote:
> Dear Jun,
>
> When $SIM and $EST on the same model file, the simulation will be run first, based on initial values, and then, afterwards, the estimation of the simulated data will be done, that’s it, no further re-simulation based on the estimated parameters will follow.
>
> So what you should do is first to analyze your observed data in an estimation model file, and then report your estimated parameters as initial values to run your simulation-estimation study; this allows you to compare estimates of real data to estimates of simulated data, so to check simulation properties of your model.
>
> I hope this helps,
>
> Elodie
>
> / Elodie Plan, PharmD, MSc, ///
>
> / PhDstudent// ///
>
> /*****************************************///
>
> /Div. of Pharmacokinetics and Drug Therapy,
> Department of Pharmaceutical Biosciences,
> Faculty of Pharmacy, //Uppsala University/
>
> / PO - Box 591 - 751 24 //Uppsala - SWEDEN/
>
> /Office +46 18 4714385 - Fax +46 18 4714003
> ------------------------------------------------------------///
>
> *From:* [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED] *On Behalf Of *Jun Shen
>
> *Sent:* Wednesday, October 15, 2008 1:01 AM
> *To:* [email protected]
> *Subject:* [NMusers] How $simulation work with $estimation
>
> Dear NMusers,
>
> I wonder how does $Simulation work with $Estimation in NONMEM exactly?
> The manual says, NONMEM will simulate DVs and replace the original DVs
>
> based on the parameter initial values. But the initial values are not final
>
> estimates. Does the simulation based on initial values make sense? Do
> $Estimation and $Simulation run alternatively? The $Simulation generates a
> set of data based on which the parameters are estimated? And then the
> predictions are made on the estimated parameters? A little confused.
>
> Appreciate any comment.
>
> Jun
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Dear Elodie/Steve/Nick,
Thank you for your replies. So we can either simulate or bootstrap to
perform a visual predictive check. I just downloaded a WFN package
(developed by Nick) which is very convenient. By doing $Sim and $Est we can
get simulated data. Now I just wonder since we also get estimated parameters
(THETA, ETA, SIGMA) from simulation or bootstrap, has anyone ever used these
parameters? I mean in addition to comparing the percentiles of the DV can we
extract any useful information by looking at those parameters? Something
like, if the estimated parameters from real dataset fall within the 95%
intervals of simulated or bootstraped data? Does such comparison make any
sense?
A following question is how we export those parameters to a table. I know I
bring up old stuff. Alison and other people have posted codes. But these
codes do not seem to work for me. I wonder if anyone has successfully used
these codes.
Attached codes I found in previous discussions.
SUBROUTINE INFN (ICALL,THETA,DATREC,INDXS,NEWIND)
DIMENSION THETA(*),DATREC(*),INDXS(*)
DOUBLE PRECISION THETA
IF (ICALL.EQ.3) THEN
DO WHILE(DATA)
IF (NEWIND.LE.1) WRITE (50,*) ETA
ENDDO
WRITE (51,*) OBJECT
WRITE (52,*) THETA
WRITE (53,*) SETHET
WRITE (54,*) OMEGA(BLOCK)
WRITE (55,*) SEOMEG(BLOCK)
WRITE (56,*) SIGMA(BLOCK)
WRITE (57,*) SESIGM(BLOCK)
WRITE (58,*) IERE,IERC
ENDIF
Quoted reply history
On Tue, Oct 14, 2008 at 8:26 PM, Nick Holford <[EMAIL PROTECTED]>wrote:
> Jun,
>
> In addition to Elodie's clear explanation of the basic process of using
> $SIM and $EST in the same control stream you may wish to know that you don't
> have to use the same parameters for simulation as those used for estimation.
> The ICALL variable has a value of 4 when NONMEM is simulating so you can do
> this:
>
> $THETA
> 1 FIX ; sim_CL
> 10 ; est_CL
>
> $PK
>
> IF (ICALL.EQ.4) THEN ; for simulation
> CL=THETA(1)
> ELSE ; for estimation
> CL=THETA(2)
> ENDIF
>
> Nick
>
>
> Elodie Plan wrote:
>
>>
>> Dear Jun,
>>
>> When $SIM and $EST on the same model file, the simulation will be run
>> first, based on initial values, and then, afterwards, the estimation of the
>> simulated data will be done, that's it, no further re-simulation based on
>> the estimated parameters will follow.
>>
>> So what you should do is first to analyze your observed data in an
>> estimation model file, and then report your estimated parameters as initial
>> values to run your simulation-estimation study; this allows you to compare
>> estimates of real data to estimates of simulated data, so to check
>> simulation properties of your model.
>>
>> I hope this helps,
>>
>> Elodie
>>
>> / Elodie Plan, PharmD, MSc, ///
>>
>> / PhDstudent// ///
>>
>> /*****************************************///
>>
>> /Div. of Pharmacokinetics and Drug Therapy,
>> Department of Pharmaceutical Biosciences,
>> Faculty of Pharmacy, //Uppsala University/
>>
>> / PO - Box 591 - 751 24 //Uppsala - SWEDEN/
>>
>> /Office +46 18 4714385 - Fax +46 18 4714003
>> ------------------------------------------------------------///
>>
>> *From:* [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
>> *On Behalf Of *Jun Shen
>> *Sent:* Wednesday, October 15, 2008 1:01 AM
>> *To:* [email protected]
>> *Subject:* [NMusers] How $simulation work with $estimation
>>
>> Dear NMusers,
>>
>> I wonder how does $Simulation work with $Estimation in NONMEM exactly?
>> The manual says, NONMEM will simulate DVs and replace the original DVs
>> based on the parameter initial values. But the initial values are not
>> final
>> estimates. Does the simulation based on initial values make sense? Do
>> $Estimation and $Simulation run alternatively? The $Simulation generates a
>> set of data based on which the parameters are estimated? And then the
>> predictions are made on the estimated parameters? A little confused.
>>
>> Appreciate any comment.
>>
>> Jun
>>
>>
> --
> Nick Holford, Dept Pharmacology & Clinical Pharmacology
> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
> Zealand
> [EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
>
>
Jun,
VPCs are created using simulation alone (i.e. without estimation) (Karlsson & Holford 2008)
If you do simulation followed by estimation this is called parametric bootstrapping (nmgosim in WFN). An alternative bootstrap method is called the non-parametric bootstrap which samples observations from the original data set (nmbs in WFN).
The non-parametric bootstrap is probably more robust than the parametric bootstrap if you have covariates in your model because it is tricky to simulate correlated covariate distributions.
The distribution of the THETA, OMEGA, SIGMA parameter estimates obtained from fittting many bootstrap replications can be used to generate confidence intervals on parameters and explore other properties of the parameter uncertainty. The bootstrap confidence intervals are more realistic than those computed from asymptotic standard errors because they dont assume normality of the uncertainty. The mean of the bootstrap distribution is probably a more robust estimate of the true population mean than the point estimate obtained from the original data.
WFN gives you all the parameter estimates from parametric or non-parametric bootstraps in a tab delimited file that can be easily read into Excel or other packages to examine the results and compute statistics from the distributions.
If you want help with getting other methods to extract parameters from NONMEM you need to learn to give useful information about what you did, what went wrong, what the EXACT error messages or unexpected results were. It may be old stuff but most nmusers are not mind readers :-)
Karlsson MO, Holford NHG. A Tutorial on Visual Predictive Checks. PAGE 17 (2008) Abstr 1434 [wwwpage-meetingorg/?abstract=1434]. 2008.
Jun Shen wrote:
> Dear Elodie/Steve/Nick,
>
> Thank you for your replies. So we can either simulate or bootstrap to perform a visual predictive check. I just downloaded a WFN package (developed by Nick) which is very convenient. By doing $Sim and $Est we can get simulated data. Now I just wonder since we also get estimated parameters (THETA, ETA, SIGMA) from simulation or bootstrap, has anyone ever used these parameters? I mean in addition to comparing the percentiles of the DV can we extract any useful information by looking at those parameters? Something like, if the estimated parameters from real dataset fall within the 95% intervals of simulated or bootstraped data? Does such comparison make any sense?
>
> A following question is how we export those parameters to a table. I know I bring up old stuff. Alison and other people have posted codes. But these codes do not seem to work for me. I wonder if anyone has successfully used these codes.
>
> Attached codes I found in previous discussions.
>
> SUBROUTINE INFN (ICALL,THETA,DATREC,INDXS,NEWIND)
> DIMENSION THETA(*),DATREC(*),INDXS(*)
> DOUBLE PRECISION THETA
>
> IF (ICALL.EQ.3) THEN
> DO WHILE(DATA)
> IF (NEWIND.LE.1) WRITE (50,*) ETA
> ENDDO
> WRITE (51,*) OBJECT
> WRITE (52,*) THETA
> WRITE (53,*) SETHET
> WRITE (54,*) OMEGA(BLOCK)
> WRITE (55,*) SEOMEG(BLOCK)
> WRITE (56,*) SIGMA(BLOCK)
> WRITE (57,*) SESIGM(BLOCK)
> WRITE (58,*) IERE,IERC
> ENDIF
>
Quoted reply history
> On Tue, Oct 14, 2008 at 8:26 PM, Nick Holford <[EMAIL PROTECTED] < mailto:[EMAIL PROTECTED]>> wrote:
>
> Jun,
>
> In addition to Elodie's clear explanation of the basic process of
> using $SIM and $EST in the same control stream you may wish to
> know that you don't have to use the same parameters for simulation
> as those used for estimation. The ICALL variable has a value of 4
> when NONMEM is simulating so you can do this:
>
> $THETA
> 1 FIX ; sim_CL
> 10 ; est_CL
>
> $PK
>
> IF (ICALL.EQ.4) THEN ; for simulation
> CL=THETA(1)
> ELSE ; for estimation
> CL=THETA(2)
> ENDIF
>
> Nick
>
> Elodie Plan wrote:
>
> Dear Jun,
>
> When $SIM and $EST on the same model file, the simulation will
> be run first, based on initial values, and then, afterwards,
> the estimation of the simulated data will be done, that's it,
> no further re-simulation based on the estimated parameters
> will follow.
>
> So what you should do is first to analyze your observed data
> in an estimation model file, and then report your estimated
> parameters as initial values to run your simulation-estimation
> study; this allows you to compare estimates of real data to
> estimates of simulated data, so to check simulation properties
> of your model.
>
> I hope this helps,
>
> Elodie
>
> / Elodie Plan, PharmD, MSc, ///
>
> / PhDstudent// ///
>
> /*****************************************///
>
> /Div. of Pharmacokinetics and Drug Therapy,
> Department of Pharmaceutical Biosciences,
> Faculty of Pharmacy, //Uppsala University/
>
> / PO - Box 591 - 751 24 //Uppsala - SWEDEN/
>
> /Office +46 18 4714385 - Fax +46 18 4714003
> ------------------------------------------------------------///
>
> *From:* [EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>
> [mailto:[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>] *On Behalf Of *Jun Shen
> *Sent:* Wednesday, October 15, 2008 1:01 AM
> *To:* [email protected] <mailto:[email protected]>
>
> *Subject:* [NMusers] How $simulation work with $estimation
>
> Dear NMusers,
>
> I wonder how does $Simulation work with $Estimation in NONMEM
> exactly?
> The manual says, NONMEM will simulate DVs and replace the
> original DVs
> based on the parameter initial values. But the initial values
> are not final
> estimates. Does the simulation based on initial values make
> sense? Do
> $Estimation and $Simulation run alternatively? The $Simulation
> generates a
> set of data based on which the parameters are estimated? And
> then the
> predictions are made on the estimated parameters? A little
> confused.
>
> Appreciate any comment.
>
> Jun
>
> -- Nick Holford, Dept Pharmacology & Clinical Pharmacology
>
> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland,
> New Zealand
> [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
> tel:+64(9)923-6730 fax:+64(9)373-7090
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford