RE: Change of NSIG or R matrix
Nick -
I think you either mis-read or mis-understood my last message.
1) you seem to be thinking of sigdigits (and NSIG for the least precise
parameter)
as an integer related to number of significant decimal digits in the parameter
estimate. Loosely speaking, this is the general
idea, but this leaves open the problem that you suggested in your example
where you claimed that at a sigdigit level of 2, a user would be somewhat
indifferent between a parameter value
of 1.0 and 1.1, a 10% difference, since these only differ by 1 in the second
significant decimal digit.
The simple interpretation of sigdigits as the (integer) number of significant
decimal digits is too vague - under it, sigdigit=2 implies a 10% precision
for estimates like 1.0,
but a 1% precision for estimates like 9.9. The use of sigdigit =
-log10(parameter precision) preserves the general idea of number of significant
decimal digits,
but makes this idea much precise.
I disagree (except as amplified in part 2 below) that what I described in my
last message is " is more like the meaning of TOL (which is used to control the
local error for the DEQ solver)"
What I described was in fact was exactly the most usual (in the numerical
analysis community) definition of sigdigits as it relates to precision of the
parameter estimate
namely that sigdigits is -log10(precision of parameter estimates). Note under
this interpretation, sigdigits is a floating point number, not an integer
NONMEM indeed reports a floating point number (not an integer) for SIGDIGITS
in it MINIMIZATION SUCCESSFUL message, e.g.
NO. OF SIG. DIGITS IN FINAL EST.: 3.4
But lurking under NSIG is in fact a tolerance, since the specification of NSIG
as a convergence criterion somewhere must be implemented as a tolerance test
involving the underlying quantities used to
estimate NSIG , so
2) I provided what I believe to be the most reasonable method of estimating
precision - i.e. inferring it from
magnitude of the objective function gradient and approximate Hessian values at
the converged parameter estimate. So here is the connection between
NSIG and a tolerance - specifying an NSIG value in fact puts a limit on how
large the quantity ( inverse(approximate hessian)* gradient) can be at the
converged value.
Again this is highly consistent with what HELP suggests when
it notes that higher SIGL's (and hence more precise gradient estimates) should
be used with
higher values of NSIG - you need a more precise gradient value if you want to
to use it in connection with showing that a parameter precision meets the NSIG
criterion . (As I noted in my
original message, the Hessian is more problematical - the best a quasi-Newton
method can do is substitute the
accumulated curvature matrix as an approximation for the Hessian. This matrix
is constructed out of sums of the tensor products of the gradient vectors from
all iterations up to the current one, and thus has roughly the same precision
as the gradient. But it is not really the same as
the Hessian - if it were, there would be no need for a COVR step to try to
estimate the real Hessian)
If the SIGL value is too low, the estimate of the parameter precision
in terms of first and second derivatives of the objective function itself
becomes too imprecise, e.g.
you may never be able to guarantee that a gradient computed with low
precision has a magnitude low enough to meet a high NSIG precision criterion.
The HELP-suggested use of a SIGL '2 or 3 times' that of the NSIG value is
consistent with the well -known square root (first derivatives) or cube root
(second derivatives) connection between the optimal step size for computing
numerical difference -based derivative values and the precision of the
evaluations of the function
that is being differentiated.
Finally, note that HELP also mentions . ' SIGDIGITS is not used by the
Monte-Carlo methods."
The Monte Carlo methods are precisely the ones which do not use explicit
gradients of the objective function,
so are unable to do compute a gradient-based precision SIGDIGITS estimate.
Quoted reply history
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Nick Holford
Sent: Wednesday, October 23, 2013 1:51 AM
To: nmusers
Subject: Re: [NMusers] Change of NSIG or R matrix
Bob,
NM-Help defines what it means by the SIGDIG estimation option.
SIGDIGITS=n
Number of significant digits required in the final parameter
estimate. SIGDIGITS is not used by the Monte-Carlo methods.
Default: 3. May also be coded NSIGDIGITS.
SIGL=n
n is used to calculate the step-size for finite difference deriv-
atives independent of the SIGDIGITS value. If n=0 or n=100 then
SIGL is ignored and SIGDIGITS is used as in versions prior to
NONMEM 7. SIGL should usually be 2 to 3 times the value of NSIG.
The number of significant digits reported is the
number of significant digits in the least-well-determined element.
The report "MINIMIZATION SUCCESSFUL" is issued when this number is no
less than the number of significant digits requested using the SIGDIG-
ITS option of the $ESTIMATION record.
NONMEM 7 has an additional estimation option (SIGL) that it used to provide
additional control for finite-difference derivatives.
Unless Bob Bauer can clarify the meaning further I still believe that the
meaning of SIGDIG when used as a convergence criterion refers to the number of
significant digits in the parameter value. What you describe is more like the
meaning of TOL (which is used to control the local error for the DEQ solver).
I have no particular interest in the accuracy of the calculation of the number
of significant digits in the parameter estimate but there is a large body of
empirical experience using SIGDIG of 3 (or more).
Furthermore, NONMEM reports parameter estimates with 3 significant digits which
I am prepared to believe more if convergence was achieved with SIGDIG of 3.