Dear NMusers,
I have questions about bootstraping and NONMEM VI assement of SEs.
I have developed a 2-compartment PK model using NMVI that converged with estimations of THETAs and OMEGAs. The predictive check simulations indicated that the model satisfactorily described the data with parameters estimated by NMVI. Then I started the assement of SEs by boostrating the data using WFN (1000 resamplings). I found similar results for THETAs, OMEGAs and SIGMAs (mixt error used). About the results of SEs, I also found similar results for the SEs of THETA1 (Volume), THETA2 (Clearance), OMEGAs and SIGMA1 (proportional part), but not for the SEs of THETA3 nor THETA4 (K12 and K21) and the SIGMA2 (residual error). I think that the actual values are more close of those obtained with bootstrap than those obtained with NMVI.
The results of the obtained SEs are described in the table below. I also performed a run with the same Input and data set with NMV and the results are also described in the table (NMV gives same results for thetas, omegas, sigmas and OFV).
SE of: NMVI values Bootstrap values NMV values
THETA1 0.148 0.154 0.154
THETA2 0.00218 0.00242 0.0227
THETA3 /*0.00063 0.0013 0.0013*/
THETA4 /*0.00036 0.0022 0.0019*/
OMEGA1 0.0094 0.0095 0.0094
OMEGA2 0.0050 0.0054 0.0051
SIGMA1 0.0063 0.0064 0.0064
SIGMA2 /* 0.675 0.946 0.832*/
My questions are:
- Why NMVI gives evaluations of SE less reliable than NMV ? and why only for THETA3, THETA4 and SIGMA 1
- for covariates determination wiht NMVI, do I need to perform bootstrap for each covariate or taking into account the decrease of omega is sufficient ?
- The value of residual error obtained with NMVI is 1.69 (SE = 0.675). The value obtained with boostrap is 1.58 (SE = 0.95), thus zero is included in IC95. How interpreting residual error including zero in IC95 with boostrap but not with NMVI. Removing SIGMA2 leads to failure of the run. Should I fix this value or leaving it with its SE ?
Bernard Royer
Pharmacology Dpt
University Hospital
Besancon, France
NMVI and SEs assessment
Hi Bernad,
Regarding differences between nm5 and 6, you can try to run both models
with different initial estimates, to see if the two minima are stable
within a nonmem version. Which minima yield the lowest OFV? Do you have
information in your data to describe a two-compartment model? What is
the correlation between the estimates of THETA3 and THETA4? You can
assess this on your 1000 sets of bootstrap parameters.
To calculate confidence interval for a parameter you use the bootstrap
parameters directly and calculate the percentiles. If you use the
bootstrap parameters to calculate SE and then use SE to calculate the
confidence interval, you are assuming that the distribution is normal
(which is wrong in this case).
Finally, regarding the covariate model: If you have few subjects in your
dataset, you may get a reduction in OMEGA which may seem relevant in
your sample, but which actually is not. To evaluate the uncertainty in
clinical relevance you can set up a criterion and evaluate for each
bootstrap sample. For example, you can plot de distribution of
(CLgenotyp1/CLgenotype2-1)/SQRT(omegaCL) for your 1000 bootstrap
samples.
Good luck!
Jakob
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Bernard ROYER
Sent: 23 April 2008 10:01
To: [email protected]
Subject: [NMusers] NMVI and SEs assessment
Dear NMusers,
I have questions about bootstraping and NONMEM VI assement of SEs.
I have developed a 2-compartment PK model using NMVI that converged with
estimations of THETAs and OMEGAs. The predictive check simulations
indicated that the model satisfactorily described the data with
parameters estimated by NMVI. Then I started the assement of SEs by
boostrating the data using WFN (1000 resamplings). I found similar
results for THETAs, OMEGAs and SIGMAs (mixt error used). About the
results of SEs, I also found similar results for the SEs of THETA1
(Volume), THETA2 (Clearance), OMEGAs and SIGMA1 (proportional part), but
not for the SEs of THETA3 nor THETA4 (K12 and K21) and the SIGMA2
(residual error). I think that the actual values are more close of those
obtained with bootstrap than those obtained with NMVI.
The results of the obtained SEs are described in the table below. I also
performed a run with the same Input and data set with NMV and the
results are also described in the table (NMV gives same results for
thetas, omegas, sigmas and OFV).
SE of: NMVI values Bootstrap values NMV values
THETA1 0.148 0.154 0.154
THETA2 0.00218 0.00242 0.0227
THETA3 0.00063 0.0013 0.0013
THETA4 0.00036 0.0022 0.0019
OMEGA1 0.0094 0.0095 0.0094
OMEGA2 0.0050 0.0054 0.0051
SIGMA1 0.0063 0.0064 0.0064
SIGMA2 0.675 0.946 0.832
My questions are:
- Why NMVI gives evaluations of SE less reliable than NMV ? and why only
for THETA3, THETA4 and SIGMA 1
- for covariates determination wiht NMVI, do I need to perform bootstrap
for each covariate or taking into account the decrease of omega is
sufficient ?
- The value of residual error obtained with NMVI is 1.69 (SE = 0.675).
The value obtained with boostrap is 1.58 (SE = 0.95), thus zero is
included in IC95. How interpreting residual error including zero in IC95
with boostrap but not with NMVI. Removing SIGMA2 leads to failure of the
run. Should I fix this value or leaving it with its SE ?
Bernard Royer
Pharmacology Dpt
University Hospital
Besancon, France
Bernard
The bootstrap method is intended to give you a reliable measurement of the
uncertainty of the estimates. The interest is then to obtain, for example, the
median and the 95% confidence interval of estimates via the 2.5th and 97.5th
quantiles of each parameter. So I think it is not really necessary to look at
the bootstrap statistics on the SEs produced by the Nonmem output (in my
experience and probably that of other users, SEs produced by Nonmem may vary
with initial values, or the $COV step can be achieved or not also depending on
initial estimates). I personally use to remove the $COV when I do a bootstrap
via WfN
Hope that it heps
Saik
----- Original Message -----
Quoted reply history
From: Bernard ROYER
To: [email protected]
Sent: Wednesday, April 23, 2008 11:01 AM
Subject: [NMusers] NMVI and SEs assessment
Dear NMusers,
I have questions about bootstraping and NONMEM VI assement of SEs.
I have developed a 2-compartment PK model using NMVI that converged with
estimations of THETAs and OMEGAs. The predictive check simulations indicated
that the model satisfactorily described the data with parameters estimated by
NMVI. Then I started the assement of SEs by boostrating the data using WFN
(1000 resamplings). I found similar results for THETAs, OMEGAs and SIGMAs (mixt
error used). About the results of SEs, I also found similar results for the SEs
of THETA1 (Volume), THETA2 (Clearance), OMEGAs and SIGMA1 (proportional part),
but not for the SEs of THETA3 nor THETA4 (K12 and K21) and the SIGMA2 (residual
error). I think that the actual values are more close of those obtained with
bootstrap than those obtained with NMVI.
The results of the obtained SEs are described in the table below. I also
performed a run with the same Input and data set with NMV and the results are
also described in the table (NMV gives same results for thetas, omegas, sigmas
and OFV).
SE of: NMVI values Bootstrap values NMV values
THETA1 0.148 0.154 0.154
THETA2 0.00218 0.00242 0.0227
THETA3 0.00063 0.0013 0.0013
THETA4 0.00036 0.0022 0.0019
OMEGA1 0.0094 0.0095 0.0094
OMEGA2 0.0050 0.0054 0.0051
SIGMA1 0.0063 0.0064 0.0064
SIGMA2 0.675 0.946 0.832
My questions are:
- Why NMVI gives evaluations of SE less reliable than NMV ? and why only for
THETA3, THETA4 and SIGMA 1
- for covariates determination wiht NMVI, do I need to perform bootstrap for
each covariate or taking into account the decrease of omega is sufficient ?
- The value of residual error obtained with NMVI is 1.69 (SE = 0.675). The
value obtained with boostrap is 1.58 (SE = 0.95), thus zero is included in
IC95. How interpreting residual error including zero in IC95 with boostrap but
not with NMVI. Removing SIGMA2 leads to failure of the run. Should I fix this
value or leaving it with its SE ?
Bernard Royer
Pharmacology Dpt
University Hospital
Besancon, France