RE: Change of NSIG or R matrix
Nick -
a) The usual definition of 'number of significant digits' is -log10(relative
precision). Thus a sigdig of 3 is a precision of 1 part in 1000, and a sigdig
of 2 corresponds to 1% precision, not 10% as in your example.
b) that being said, the sigdigs in the parameters reported by NONMEM need to be
taken with a grain of salt - they probably represent best case, 'speed of
light' type numbers where the real precision may be considerably worse. I do
not know specifically how they are computed, but my guess is that it is based
on the fact that in a converged problem, the relative gradient has been driven
below some specified tolerance. One can then infer precision from the
condition number of the Hessian of the overall objective function and the
actual relative gradients. But NONMEM uses a quasi-Newton method -
there is no Hessian available to the method, but only a stand-in accumulated
curvature matrix (a 'pseudo Hessian') that is usually much better conditioned
than the actual Hessian. The only thing that can really be concluded is that,
at the moment the top level iteration is stopped and convergence declared based
on the relative gradient, the next iteration , if it were done, would not
change the parameter estimates by more than the reported sigdig value. This is
quite a different conclusion than reported parameter estimates are with
sigdigits of the 'true' values.
c) I know you have often argued that the failure of a covariance step has
little or no evidential value for determining whether the minimization step was
'successful', and I generally agree with you.
But the failure of the covariance step does mean that the Hessian could not be
numerically estimated at all (failed the positive definiteness test). This
does provide some additional evidence that one should be even more skeptical of
the reported sigdig values of the parameter estimates.
Quoted reply history
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Nick Holford
Sent: Tuesday, October 22, 2013 4:45 AM
To: [email protected]
Subject: Re: [NMusers] Change of NSIG or R matrix
Xinting,
First of all 'successful minimization' has nothing to do with a good model.
NONMEM's internal decision to declare success or termination is often a
pseudo-random choice. If you look at the sigdigs of the estimate you will
typically find that the lowest value is 2.9 and many others are greater than 5.
This gives you a clue to which parameters are well determined and which are
less well known. It is a not a YES/NO decision.
Second, NSIG determines the number of significant digits in the parameter
estimates. If you choose a number less than 3 then it means you don't care if
the answer is 10.1 or 10.9. They both have 2 sig digs but the estimates differ
by nearly 10%. There is a large body of empirical literature that has relied on
NSIG=3 (or more). I do not see any reason to ignore this in order to get a
meaningless "minimization successful" message from a random number generator.
I look forward to hearing from "many" to understand why they believe that
"minimization successful" indicates that the model results are somehow better
even though the parameter estimates have hardly any significant digits.
Nick
On 22/10/2013 9:17 p.m., Xinting Wang wrote:
> Dear Nick,
>
> Thank you very much for your suggestion. Could you explain a little
> bit about the statement regarding NSIG < 3? I seem to remember that
> many suggested to use a smaller NSIG to get a successful minimization.
>
> Dear Leonid,
>
> I read about the recommendation of SIGL, NSIG and TOL, but I am not
> quite familiar with the use of these options in subroutine ADVAN4. If
> I set SIGL a fixed value, let's say 12, and NSIG 3, does this mean I
> also have to identify a value for TOL in $subroutine? I appreciate
> your help very much.
>
> Thank you both.
>
> Regards
>
>
>
> On 8 October 2013 21:59, Leonid Gibiansky <[email protected]
> <mailto:[email protected]>> wrote:
>
> Yes, it should be fine to use S matrix if you cannot get default
> to run, and use NSIG larger or smaller than default value of 3
> (although this is not guaranteed, usually NSIG does not change the
> OF value or parameter estimates in any significant way). Note that
> Nonmem manual recommends that SIGL >= 3*NSIG, TOL >= SIGL.
> Separate SIGL can be set on COV step, and it is recommended that
> SIGL >= 4*NSIG on COV step. In real life I've seen many examples
> where larger NSIG and SIGL resulted in successful COV step, and
> also many examples when default values were better (in getting COV
> step). UNCONDITIONAL on COV step allows you to run COV even when
> minimization ended with some error.
>
> Contrary to Nick's experience, I found that COV step is useful as
> it reveals which of the model parameters are poorly estimated, and
> that CI based on SE are usually quite good and are in a general
> agreement with the bootstrap CI, but it may depend on the problem.
>
> Leonid
>
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com http://www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
>
>
>
>
> On 10/8/2013 3:57 AM, Xinting Wang wrote:
>
> Dear all,
>
> I have a naive question regarding the modeling building process in
> NONMEM. With more and more covariates added in the model, I
> often come
> across an error message saying that "ERROR 134", or R MATRIX
> SINGULAR.
>
> After searching from the internet, I learned that changing NSIG in
> $ESTIMATION and MATRIX=S in $COV would be helpful for both
> problems
> respectively. And from my own experience, it dose help with
> the modeling
> building.
>
> However, my concern is, I used different NSIG and MATRIX in
> the previous
> steps. Is it proper to use different NSIGs and MATRICE in a
> single model
> building? If not, could you please explain this a little bit?
>
> Thank you in advance!
>
> Best Regards
> --
> Xinting
> Wang
>
>
>
>
> --
> Xinting
--
Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical
Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park Rd,Private Bag
92019,Auckland,New Zealand
office:+64(9)923-6730 mobile:NZ +64(21)46 23 53
email: [email protected]
http://holford.fmhs.auckland.ac.nz/
Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics
and Pharmacodynamics. 2013;40:369-76
http://link.springer.com/article/10.1007/s10928-013-9316-2
Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and
adults. J Pharm Sci. 2013:
http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract
Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. 2013;2:
http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html
Holford NHG. Clinical pharmacology = disease progression + drug action. British
Journal of Clinical Pharmacology. 2013:
http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract