RE: Change of NSIG or R matrix

From: Bob Leary Date: October 23, 2013 technical Source: mail-archive.com
Nick - I think you either mis-read or mis-understood my last message. 1) you seem to be thinking of sigdigits (and NSIG for the least precise parameter) as an integer related to number of significant decimal digits in the parameter estimate. Loosely speaking, this is the general idea, but this leaves open the problem that you suggested in your example where you claimed that at a sigdigit level of 2, a user would be somewhat indifferent between a parameter value of 1.0 and 1.1, a 10% difference, since these only differ by 1 in the second significant decimal digit. The simple interpretation of sigdigits as the (integer) number of significant decimal digits is too vague - under it, sigdigit=2 implies a 10% precision for estimates like 1.0, but a 1% precision for estimates like 9.9. The use of sigdigit = -log10(parameter precision) preserves the general idea of number of significant decimal digits, but makes this idea much precise. I disagree (except as amplified in part 2 below) that what I described in my last message is " is more like the meaning of TOL (which is used to control the local error for the DEQ solver)" What I described was in fact was exactly the most usual (in the numerical analysis community) definition of sigdigits as it relates to precision of the parameter estimate namely that sigdigits is -log10(precision of parameter estimates). Note under this interpretation, sigdigits is a floating point number, not an integer NONMEM indeed reports a floating point number (not an integer) for SIGDIGITS in it MINIMIZATION SUCCESSFUL message, e.g. NO. OF SIG. DIGITS IN FINAL EST.: 3.4 But lurking under NSIG is in fact a tolerance, since the specification of NSIG as a convergence criterion somewhere must be implemented as a tolerance test involving the underlying quantities used to estimate NSIG , so 2) I provided what I believe to be the most reasonable method of estimating precision - i.e. inferring it from magnitude of the objective function gradient and approximate Hessian values at the converged parameter estimate. So here is the connection between NSIG and a tolerance - specifying an NSIG value in fact puts a limit on how large the quantity ( inverse(approximate hessian)* gradient) can be at the converged value. Again this is highly consistent with what HELP suggests when it notes that higher SIGL's (and hence more precise gradient estimates) should be used with higher values of NSIG - you need a more precise gradient value if you want to to use it in connection with showing that a parameter precision meets the NSIG criterion . (As I noted in my original message, the Hessian is more problematical - the best a quasi-Newton method can do is substitute the accumulated curvature matrix as an approximation for the Hessian. This matrix is constructed out of sums of the tensor products of the gradient vectors from all iterations up to the current one, and thus has roughly the same precision as the gradient. But it is not really the same as the Hessian - if it were, there would be no need for a COVR step to try to estimate the real Hessian) If the SIGL value is too low, the estimate of the parameter precision in terms of first and second derivatives of the objective function itself becomes too imprecise, e.g. you may never be able to guarantee that a gradient computed with low precision has a magnitude low enough to meet a high NSIG precision criterion. The HELP-suggested use of a SIGL '2 or 3 times' that of the NSIG value is consistent with the well -known square root (first derivatives) or cube root (second derivatives) connection between the optimal step size for computing numerical difference -based derivative values and the precision of the evaluations of the function that is being differentiated. Finally, note that HELP also mentions . ' SIGDIGITS is not used by the Monte-Carlo methods." The Monte Carlo methods are precisely the ones which do not use explicit gradients of the objective function, so are unable to do compute a gradient-based precision SIGDIGITS estimate.
Quoted reply history
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Wednesday, October 23, 2013 1:51 AM To: nmusers Subject: Re: [NMusers] Change of NSIG or R matrix Bob, NM-Help defines what it means by the SIGDIG estimation option. SIGDIGITS=n Number of significant digits required in the final parameter estimate. SIGDIGITS is not used by the Monte-Carlo methods. Default: 3. May also be coded NSIGDIGITS. SIGL=n n is used to calculate the step-size for finite difference deriv- atives independent of the SIGDIGITS value. If n=0 or n=100 then SIGL is ignored and SIGDIGITS is used as in versions prior to NONMEM 7. SIGL should usually be 2 to 3 times the value of NSIG. The number of significant digits reported is the number of significant digits in the least-well-determined element. The report "MINIMIZATION SUCCESSFUL" is issued when this number is no less than the number of significant digits requested using the SIGDIG- ITS option of the $ESTIMATION record. NONMEM 7 has an additional estimation option (SIGL) that it used to provide additional control for finite-difference derivatives. Unless Bob Bauer can clarify the meaning further I still believe that the meaning of SIGDIG when used as a convergence criterion refers to the number of significant digits in the parameter value. What you describe is more like the meaning of TOL (which is used to control the local error for the DEQ solver). I have no particular interest in the accuracy of the calculation of the number of significant digits in the parameter estimate but there is a large body of empirical experience using SIGDIG of 3 (or more). Furthermore, NONMEM reports parameter estimates with 3 significant digits which I am prepared to believe more if convergence was achieved with SIGDIG of 3.
Oct 08, 2013 Xinting Wang Change of NSIG or R matrix
Oct 08, 2013 Nick Holford Re: Change of NSIG or R matrix
Oct 08, 2013 Leonid Gibiansky Re: Change of NSIG or R matrix
Oct 22, 2013 Xinting Wang Re: Change of NSIG or R matrix
Oct 22, 2013 Nick Holford Re: Change of NSIG or R matrix
Oct 22, 2013 Bob Leary RE: Change of NSIG or R matrix
Oct 22, 2013 Leonid Gibiansky Re: Change of NSIG or R matrix
Oct 23, 2013 Nick Holford Re: Change of NSIG or R matrix
Oct 23, 2013 Mark Sale RE: Change of NSIG or R matrix
Oct 23, 2013 Bob Leary RE: Change of NSIG or R matrix
Oct 25, 2013 Mark Sale RE: Change of NSIG or R matrix