Hi,
I'm interested in generating samples from the asymptotic sampling distribution
of population parameter estimates from a published PKPOP model fitted with
NONMEM. By definition, parameter estimates are asymptotically (multivariate)
normally distributed (unconstrained optimization) with mean M and covariance C,
where M is the vector of parameter estimates and C is the covariance matrix of
estimates (returned by $COV and available in the lst file).
Consider the 2 models below:
Model 1:
TVCL = THETA(1)
CL = TVCL*EXP(ETA(1))
Model 2:
TVCL = EXP(THETA(1))
CL = TVCL*EXP(ETA(1))
It is clear that model 1 and model 2 will provide exactly the same fit.
However, although in both cases the standard error of estimates (SE) will refer
to THETA(1), the asymptotic sampling distribution of TVCL will be normal in
model 1 while it will be lognormal in model 2. Therefore if one is interested
in generating random samples from the asymptotic distribution of TVCL, some of
these samples might be negative in model 1 while they'll remain nicely positive
in model 2. The same would happen with bounds of (asymptotic) confidence
intervals: in model 1 the lower bound of a 95% confidence interval for TVCL
might be negative (unrealistic) while it would remain positive in model 2.
This has obviously no impact for point estimates or even confidence intervals
constructed via non-parametric bootstrap since boundary constraints can be
placed on parameters in NONMEM. But what if one is interested in the asymptotic
covariance matrix of estimates returned by $COV? The asymptotic sampling
distribution of parameter estimates is (multivariate) normal only if the
optimization is unconstrained! Doesn't this then speak in favour of model 2
over model 1? Or does NONMEM take care of it and returns the asymptotic SE of
THETA(1) in model 1 on the log-scale (when boundary constraints are placed on
the parameter)?
Thanks,
Aziz Chaouch
Standard errors of estimates for strictly positive parameters
Dear Aziz,
NM does not return the asymptotic SE of THETA(1) in model 1 on the log-scale. So I would use model 2.
With best regards / Mit freundlichen Gren / Cordialement
Pascal
Quoted reply history
From: owner-nmusers_at_globomaxnm.com [mailto:owner-nmusers_at_globomaxnm.com] On Behalf Of Chaouch Aziz
Sent: 11 February 2015 17:22
To: nmusers_at_globomaxnm.com
Subject: [NMusers] Standard errors of estimates for strictly positive parameters
Hi,
I'm interested in generating samples from the asymptotic sampling distribution of population parameter estimates from a published PKPOP model fitted with NONMEM. By definition, parameter estimates are asymptotically (multivariate) normally distributed (unconstrained optimization) with mean M and covariance C, where M is the vector of parameter estimates and C is the covariance matrix of estimates (returned by $COV and available in the lst file).
Consider the 2 models below:
Model 1:
TVCL = THETA(1)
CL = TVCL*EXP(ETA(1))
Model 2:
TVCL = EXP(THETA(1))
CL = TVCL*EXP(ETA(1))
It is clear that model 1 and model 2 will provide exactly the same fit. However, although in both cases the standard error of estimates (SE) will refer to THETA(1), the asymptotic sampling distribution of TVCL will be normal in model 1 while it will be lognormal in model 2. Therefore if one is interested in generating random samples from the asymptotic distribution of TVCL, some of these samples might be negative in model 1 while they'll remain nicely positive in model 2. The same would happen with bounds of (asymptotic) confidence intervals: in model 1 the lower bound of a 95% confidence interval for TVCL might be negative (unrealistic) while it would remain positive in model 2.
This has obviously no impact for point estimates or even confidence intervals constructed via non-parametric bootstrap since boundary constraints can be placed on parameters in NONMEM. But what if one is interested in the asymptotic covariance matrix of estimates returned by $COV? The asymptotic sampling distribution of parameter estimates is (multivariate) normal only if the optimization is unconstrained! Doesn't this then speak in favour of model 2 over model 1? Or does NONMEM take care of it and returns the asymptotic SE of THETA(1) in model 1 on the log-scale (when boundary constraints are placed on the parameter)?
Thanks,
Aziz Chaouch
This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.
Click http://www.merckgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.
Dear Aziz,
NM does not return the asymptotic SE of THETA(1) in model 1 on the log-scale.
So I would use model 2.
With best regards / Mit freundlichen Grüßen / Cordialement
Pascal
Quoted reply history
From: [email protected] [mailto:[email protected]] On
Behalf Of Chaouch Aziz
Sent: 11 February 2015 17:22
To: [email protected]
Subject: [NMusers] Standard errors of estimates for strictly positive parameters
Hi,
I'm interested in generating samples from the asymptotic sampling distribution
of population parameter estimates from a published PKPOP model fitted with
NONMEM. By definition, parameter estimates are asymptotically (multivariate)
normally distributed (unconstrained optimization) with mean M and covariance C,
where M is the vector of parameter estimates and C is the covariance matrix of
estimates (returned by $COV and available in the lst file).
Consider the 2 models below:
Model 1:
TVCL = THETA(1)
CL = TVCL*EXP(ETA(1))
Model 2:
TVCL = EXP(THETA(1))
CL = TVCL*EXP(ETA(1))
It is clear that model 1 and model 2 will provide exactly the same fit.
However, although in both cases the standard error of estimates (SE) will refer
to THETA(1), the asymptotic sampling distribution of TVCL will be normal in
model 1 while it will be lognormal in model 2. Therefore if one is interested
in generating random samples from the asymptotic distribution of TVCL, some of
these samples might be negative in model 1 while they'll remain nicely positive
in model 2. The same would happen with bounds of (asymptotic) confidence
intervals: in model 1 the lower bound of a 95% confidence interval for TVCL
might be negative (unrealistic) while it would remain positive in model 2.
This has obviously no impact for point estimates or even confidence intervals
constructed via non-parametric bootstrap since boundary constraints can be
placed on parameters in NONMEM. But what if one is interested in the asymptotic
covariance matrix of estimates returned by $COV? The asymptotic sampling
distribution of parameter estimates is (multivariate) normal only if the
optimization is unconstrained! Doesn't this then speak in favour of model 2
over model 1? Or does NONMEM take care of it and returns the asymptotic SE of
THETA(1) in model 1 on the log-scale (when boundary constraints are placed on
the parameter)?
Thanks,
Aziz Chaouch
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient, you
must not copy this message or attachment or disclose the contents to any other
person. If you have received this transmission in error, please notify the
sender immediately and delete the message and any attachment from your system.
Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept
liability for any omissions or errors in this message which may arise as a
result of E-Mail-transmission or for damages resulting from any unauthorized
changes of the content of this message and any attachment thereto. Merck KGaA,
Darmstadt, Germany and any of its subsidiaries do not guarantee that this
message is free of viruses and does not accept liability for any damages caused
by any virus transmitted therewith.
Click http://www.merckgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Hi Aziz,
Just some comments off the top of my head in a quite informal way: I'm not
really sure that these are the same problem because they dont start with the
same information in the form of parameter constraints. In model 1 you are
asking the optimizer for the unconstrained maximum likelihood solution for
TVCL. OK, this is reasonable in a lot of situations, but not necessairily in
all situations.
In model 2 you add information by forcing TVCL and CL to be positive. If you
think of the optimal solution as some point in N-dimensional space which has to
be searched for, in model 2 you are saying “dont even look in the space where
TVCL or CL is negative”. Even stronger, in model 2 you are also saying “dont
even get close to zero” because the log-normal distribution vanishes towards
zero.
Which solution of these is best for some particular application depends on a
lot of things. One of the things I would think about in this situation is
whether or not my a priori beliefs match with the structual constraints of the
model. Do I really think that the “true” CL could be zero? If yes, then model 2
is hard to defend in that case.
You description of your situation regarding standard errors is a part of the
same thing. When you extrapolate standard errors into low-probability areas you
are checking the boundaries of the probability area. It should not be suprising
that model 1 might tell you that CL is negative since this was part of the
solution space which you allowed. With model 2 your model structure says “dont
even look there”
In short, although these two models might look similar, I think they are really
quite different. This becomes most clear when you consider the low-probability
space.
Sorry for the vauge language.
Warm regards,
Douglas
Quoted reply history
________________________________________
From: [email protected] [[email protected]] on behalf of
Chaouch Aziz [[email protected]]
Sent: Wednesday, February 11, 2015 5:21 PM
To: [email protected]
Subject: [NMusers] Standard errors of estimates for strictly positive parameters
Hi,
I'm interested in generating samples from the asymptotic sampling distribution
of population parameter estimates from a published PKPOP model fitted with
NONMEM. By definition, parameter estimates are asymptotically (multivariate)
normally distributed (unconstrained optimization) with mean M and covariance C,
where M is the vector of parameter estimates and C is the covariance matrix of
estimates (returned by $COV and available in the lst file).
Consider the 2 models below:
Model 1:
TVCL = THETA(1)
CL = TVCL*EXP(ETA(1))
Model 2:
TVCL = EXP(THETA(1))
CL = TVCL*EXP(ETA(1))
It is clear that model 1 and model 2 will provide exactly the same fit.
However, although in both cases the standard error of estimates (SE) will refer
to THETA(1), the asymptotic sampling distribution of TVCL will be normal in
model 1 while it will be lognormal in model 2. Therefore if one is interested
in generating random samples from the asymptotic distribution of TVCL, some of
these samples might be negative in model 1 while they'll remain nicely positive
in model 2. The same would happen with bounds of (asymptotic) confidence
intervals: in model 1 the lower bound of a 95% confidence interval for TVCL
might be negative (unrealistic) while it would remain positive in model 2.
This has obviously no impact for point estimates or even confidence intervals
constructed via non-parametric bootstrap since boundary constraints can be
placed on parameters in NONMEM. But what if one is interested in the asymptotic
covariance matrix of estimates returned by $COV? The asymptotic sampling
distribution of parameter estimates is (multivariate) normal only if the
optimization is unconstrained! Doesn't this then speak in favour of model 2
over model 1? Or does NONMEM take care of it and returns the asymptotic SE of
THETA(1) in model 1 on the log-scale (when boundary constraints are placed on
the parameter)?
Thanks,
Aziz Chaouch
________________________________
Dear Douglas, dear Pascal,
Thanks a lot for your answers. I guess the main point here is constrained vs
unconstrained optimization as the asymptotic covariance matrix of estimates (as
returned by $COV) is "well defined" only in the latter case. When fitting model
1, one would normally constrain THETA(1) to be positive by using something like:
$THETA
(0, 15, 50) ; TVCL
In this situation I wonder whether it makes sense at all to consider the output
of $COV. It seems model 2 would be here preferable (unconstrained
optimization). If model 1 is fitted without boundary constraints on THETA(1),
the covariance matrix of estimates may have "some" meaning but the optimization
in NONMEM is then likely to crash if it encounters a negative value at some
point, which again speaks somehow in favor of model 2 (unless one is not
interested in the output of $COV).
Now what about $OMEGA? Here NONMEM knows that these are variances and therefore
we do not need to explicitly (i.e. manually) place boundary constraints on the
diagonal elements of the omega matrix. However something must account for it
internally. The covariance matrix of estimates returned by $COV also contains
elements that refer to omega so I'm unsure how these are treated. For diagonal
elements of the omega matrix, does NONMEM optimize log(omega) or omega? Or does
it uses a Cholesky decomposition of the Omega matrix and optimize elements on
that scale? Again, unless the optimization on omega is unconstrained, can we
really trust the output of $COV? Basically the question here is how would you
construct an asymptotic 95% confidence interval for a diagonal element of Omega
(i.e. a variance) based on the information from the covariance matrix of
estimates?
The covariance matrix of estimate is of importance to me because I'm
considering published studies and I do not have access to the data so I cannot
refit the model with an alternative parametrization. Results from $COV (in lst
file when available from the authors) is then the only available piece of
information about the uncertainty of the estimation process.
Kind regards,
Aziz Chaouch
________________________________________
Quoted reply history
De : Eleveld, DJ [mailto:[email protected]]
Envoyé : mercredi, 11. février 2015 22:26
À : Chaouch Aziz; [email protected]
Objet : RE: Standard errors of estimates for strictly positive parameters
Hi Aziz,
Just some comments off the top of my head in a quite informal way: I'm not
really sure that these are the same problem because they dont start with the
same information in the form of parameter constraints. In model 1 you are
asking the optimizer for the unconstrained maximum likelihood solution for
TVCL. OK, this is reasonable in a lot of situations, but not necessairily in
all situations.
In model 2 you add information by forcing TVCL and CL to be positive. If you
think of the optimal solution as some point in N-dimensional space which has to
be searched for, in model 2 you are saying "dont even look in the space where
TVCL or CL is negative". Even stronger, in model 2 you are also saying "dont
even get close to zero" because the log-normal distribution vanishes towards
zero.
Which solution of these is best for some particular application depends on a
lot of things. One of the things I would think about in this situation is
whether or not my a priori beliefs match with the structual constraints of the
model. Do I really think that the "true" CL could be zero? If yes, then model 2
is hard to defend in that case.
You description of your situation regarding standard errors is a part of the
same thing. When you extrapolate standard errors into low-probability areas you
are checking the boundaries of the probability area. It should not be suprising
that model 1 might tell you that CL is negative since this was part of the
solution space which you allowed. With model 2 your model structure says "dont
even look there"
In short, although these two models might look similar, I think they are really
quite different. This becomes most clear when you consider the low-probability
space.
Sorry for the vauge language.
Warm regards,
Douglas
________________________________________
De : [email protected] [mailto:[email protected]]
Envoyé : mercredi, 11. février 2015 18:30
À : Chaouch Aziz; [email protected]
Objet : RE: Standard errors of estimates for strictly positive parameters
Dear Aziz,
NM does not return the asymptotic SE of THETA(1) in model 1 on the log-scale.
So I would use model 2.
With best regards / Mit freundlichen Grüßen / Cordialement
Pascal
________________________________________
From: [email protected] [[email protected]] on behalf of
Chaouch Aziz [[email protected]]
Sent: Wednesday, February 11, 2015 5:21 PM
To: [email protected]
Subject: [NMusers] Standard errors of estimates for strictly positive parameters
Hi,
I'm interested in generating samples from the asymptotic sampling distribution
of population parameter estimates from a published PKPOP model fitted with
NONMEM. By definition, parameter estimates are asymptotically (multivariate)
normally distributed (unconstrained optimization) with mean M and covariance C,
where M is the vector of parameter estimates and C is the covariance matrix of
estimates (returned by $COV and available in the lst file).
Consider the 2 models below:
Model 1:
TVCL = THETA(1)
CL = TVCL*EXP(ETA(1))
Model 2:
TVCL = EXP(THETA(1))
CL = TVCL*EXP(ETA(1))
It is clear that model 1 and model 2 will provide exactly the same fit.
However, although in both cases the standard error of estimates (SE) will refer
to THETA(1), the asymptotic sampling distribution of TVCL will be normal in
model 1 while it will be lognormal in model 2. Therefore if one is interested
in generating random samples from the asymptotic distribution of TVCL, some of
these samples might be negative in model 1 while they'll remain nicely positive
in model 2. The same would happen with bounds of (asymptotic) confidence
intervals: in model 1 the lower bound of a 95% confidence interval for TVCL
might be negative (unrealistic) while it would remain positive in model 2.
This has obviously no impact for point estimates or even confidence intervals
constructed via non-parametric bootstrap since boundary constraints can be
placed on parameters in NONMEM. But what if one is interested in the asymptotic
covariance matrix of estimates returned by $COV? The asymptotic sampling
distribution of parameter estimates is (multivariate) normal only if the
optimization is unconstrained! Doesn't this then speak in favour of model 2
over model 1? Or does NONMEM take care of it and returns the asymptotic SE of
THETA(1) in model 1 on the log-scale (when boundary constraints are placed on
the parameter)?
Thanks,
Aziz Chaouch
________________________________
Dear Aziz -
The approximate likelihood methods in NONMEM such as FO, FOCE,and LAPLACE
optimize an objective function than is parameterized internally
by the Cholesky factor L of Omega, regardless of whether the matrix is diagonal
(the EM -based methods do something considerably different and work directly
with Omega rather than
the Cholesky factor.)
Thus for the approximate likelihood methods, the SE's computed internally by
$COV from the Hessian or Sandwich or Fisher score methods
are first computed with respect to these Cholesky parameters , and then the
corresponding SE's of the full Omega=LL' are computed by a 'propagation of
errors' approach
which skews the results, particularly if the SE's are large. Thus in a sense
regarding your dilemma about whether Model 1 or Model 2 is better with respect
to applicability of $COV results, one answer is that both are fundamentally
distorted by the propagation of errors method with respect to the Omega
elements.
But regarding your fundamental question 'can we trust the output of $COV '-
all of this makes very little difference. Standard errors computed by $COV are
inherently dubious - the applicability of the usual asymptotic arguments is
very questionable for the types/sizes of data sets we often deal with.
As Lewis Sheiner used to say of these results, 'they are not worth the
electrons used to compute them'. They are the best we can do for the level
of computational investment put into them -
If you want something better, try a bootstrap or profiling method.
________________________________________
I think we are back to the discussion of usefulness of SE/RSE provided by Nonmem (search in archives will recover many e-mails on the subject). I am reluctant to disagree with Lewis Sheiner (if this indeed was his citation that was not taken out of context) but SEs are very useful (not perfect but useful) as an indicator of identifiability of the problem and some measure of precision. One cannot do bootstrap on each and every step of the modeling (moreover, bootstrap CI are not that different from asymptotic CI, especially for well-estimated parameters). Profiling is also local, and unlikely to give something significantly different from the Nonmem SE results unless RSEs are very large (and profiling is as time-consuming as bootstrap). So SEs serve as a very useful indicator whether data support the model.
Note that Nonmem SEs are local; they rely on approximation of the OF surface, essentially, give the curvature of this surface, so they do not take into account any constrains. In this sense, form of the model (CL=THETA(1) or CL=log(THETA(2)) just define the rule of extrapolation beyond the locality of the estimate. For practical purposes, SEs of these two representations are related (approximately) as
SE(THETA2)=SE(THETA(1))/THETA(1)
(propagation of error rule). Given SE(THETA(1)) and THETA(1) you can estimate SE(THETA(2)) and simulate accordingly.
Thanks
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 2/12/2015 9:04 AM, Bob Leary wrote:
> Dear Aziz -
> The approximate likelihood methods in NONMEM such as FO, FOCE,and LAPLACE
> optimize an objective function than is parameterized internally
> by the Cholesky factor L of Omega, regardless of whether the matrix is diagonal
> (the EM -based methods do something considerably different and work directly
> with Omega rather than
> the Cholesky factor.)
>
> Thus for the approximate likelihood methods, the SE's computed internally by
> $COV from the Hessian or Sandwich or Fisher score methods
> are first computed with respect to these Cholesky parameters , and then the
> corresponding SE's of the full Omega=LL' are computed by a 'propagation of
> errors' approach
> which skews the results, particularly if the SE's are large. Thus in a sense
> regarding your dilemma about whether Model 1 or Model 2 is better with respect
> to applicability of $COV results, one answer is that both are fundamentally
> distorted by the propagation of errors method with respect to the Omega
> elements.
>
> But regarding your fundamental question 'can we trust the output of $COV '-
> all of this makes very little difference. Standard errors computed by $COV are
> inherently dubious - the applicability of the usual asymptotic arguments is
> very questionable for the types/sizes of data sets we often deal with.
> As Lewis Sheiner used to say of these results, 'they are not worth the
> electrons used to compute them'. They are the best we can do for the level
> of computational investment put into them -
> If you want something better, try a bootstrap or profiling method.
>
> ________________________________________
>
>
Hi,
The original quote about electrons comes from a remark I made in 1999 on nmusers.
http://www.cognigencorp.com/nonmem/nm/99nov121999.html
Lewis Sheiner agreed in the same thread. Thanks to the wonders of living on a sphere Lewis appears to agree with me the day before I made the comment :-)
My comments then about the relative merits of asymptotic SEs ("some kind of rough diagnostic"), log-likelihood profiling ("may require reparameterization") and non-parametric bootstrap ("reasonable values for a confidence interval") remain unchanged. The only change I would make is that I currently find 100 bootstrap replicates is good enough for model selection and evaluation. With the availability of grid computing bootstrap runs can be more readily included in the model selection strategy by identifying parameters which are poorly estimable.
Most published papers I have read recently and most presentations at PAGE use bootstrap estimates to describe uncertainty as part of model evaluation.
Unlike Leonid my experience is that bootstrap confidence intervals are often asymmetrical and do not agree with asymptotic SE predictions (e.g. see Matthews 2004). This may be because I deal with more non-linear problems. There is also the problem that when models get sufficiently complex to be interesting NONMEM is often unable to calculate an asymptotic SE (Holford's Rule -- http://www.cognigencorp.com/nonmem/current/2008-July/0017.html ).
Matthews I, Kirkpatrick C, Holford N. Quantitative justification for target concentration intervention--parameter variability and predictive performance using population pharmacokinetic models for aminoglycosides. Br J Clin Pharmacol. 2004;58(1):8-19.
Best wishes,
Nick
Quoted reply history
On 13/02/2015 4:51 a.m., Bob Leary wrote:
> Hi Leonid -
> a) I recall the general occasion of the Lewis Sheiner quote - it was at the
> PAGE meeting in Paris in 2002 at the end of a talk I gave there,
> but not the specific question that inspired it (but I did show a profiling
> graph there, so maybe it was related to that).
> b) Profiling is NOT a local method - that's the whole point of why it is
> useful, at least for single parameters of particular interest.
> It explores the objective function surface or actually the optimal objective
> function and parameters conditioned on a selected parameter moving
> along a line that extends to points that are potentially far away from the
> original optimum. If you are willing to focus on just a few parameters
> of interest, it actually avoids some of the potential pitfalls of bootstrapping
> . Moreover, it is theoretically quite sound (no asymptotic arguments
> required) as long as the objective function is really the log likelihood or a
> decent approximation to it.
>
> -----Original Message-----
> From: Leonid Gibiansky [mailto:[email protected]]
> Sent: Thursday, February 12, 2015 10:09 AM
> To: Bob Leary; Chaouch Aziz; Eleveld, DJ; [email protected]
> Cc: [email protected]
> Subject: Re: [NMusers] RE: Standard errors of estimates for strictly positive
> parameters
>
> I think we are back to the discussion of usefulness of SE/RSE provided by
> Nonmem (search in archives will recover many e-mails on the subject).
> I am reluctant to disagree with Lewis Sheiner (if this indeed was his citation
> that was not taken out of context) but SEs are very useful (not perfect but
> useful) as an indicator of identifiability of the problem and some measure of
> precision. One cannot do bootstrap on each and every step of the modeling
> (moreover, bootstrap CI are not that different from asymptotic CI, especially
> for well-estimated parameters). Profiling is also local, and unlikely to give
> something significantly different from the Nonmem SE results unless RSEs are
> very large (and profiling is as time-consuming as bootstrap). So SEs serve as a
> very useful indicator whether data support the model.
>
> Note that Nonmem SEs are local; they rely on approximation of the OF surface,
> essentially, give the curvature of this surface, so they do not take into
> account any constrains. In this sense, form of the model
> (CL=THETA(1) or CL=log(THETA(2)) just define the rule of extrapolation beyond
> the locality of the estimate. For practical purposes, SEs of these two
> representations are related (approximately) as
>
> SE(THETA2)=SE(THETA(1))/THETA(1)
>
> (propagation of error rule). Given SE(THETA(1)) and THETA(1) you can estimate
> SE(THETA(2)) and simulate accordingly.
>
> Thanks
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com
> tel: (301) 767 5566
>
> On 2/12/2015 9:04 AM, Bob Leary wrote:
>
> > Dear Aziz -
> > The approximate likelihood methods in NONMEM such as FO, FOCE,and
> > LAPLACE optimize an objective function than is parameterized
> > internally by the Cholesky factor L of Omega, regardless of whether
> > the matrix is diagonal (the EM -based methods do something
> > considerably different and work directly with Omega rather than the
> > Cholesky factor.)
> >
> > Thus for the approximate likelihood methods, the SE's computed
> > internally by $COV from the Hessian or Sandwich or Fisher score
> > methods are first computed with respect to these Cholesky parameters , and then
> > the corresponding SE's of the full Omega=LL' are computed by a 'propagation of
> > errors' approach which skews the results, particularly if the SE's are large.
> > Thus in a sense regarding your dilemma about whether Model 1 or Model 2 is
> > better with respect to applicability of $COV results, one answer is that both
> > are fundamentally distorted by the propagation of errors method with respect
> > to the Omega elements.
> >
> > But regarding your fundamental question 'can we trust the output of $COV '-
> > all of this makes very little difference. Standard errors computed by $COV are
> > inherently dubious - the applicability of the usual asymptotic arguments is
> > very questionable for the types/sizes of data sets we often deal with.
> > As Lewis Sheiner used to say of these results, 'they are not worth the
> > electrons used to compute them'. They are the best we can do for the level
> > of computational investment put into them -
> > If you want something better, try a bootstrap or profiling method.
> >
> > ________________________________________
> >
> >