Hi,
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
I cannot seem to find any references to the method by which this number
is computed.
Xavier Woot de Trixhe, ir.
Scientist, AM&S
Clinical Pharmacology
Tel: +32 (0)14 60 29 70
Janssen Research & Development
Turnoudseweg 30
2340 Beerse, Belgium
0ITERATION NO.: 13 OBJECTIVE VALUE: -14134.5071734877 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 102
PARAMETER: -2.2522E-01 -3.6013E-01 -4.2825E-01 -2.6114E-01 -1.1966E-01
GRADIENT: 7.0164E-01 -5.8463E-01 6.5547E-01 -2.4262E-01 4.5721E-01
0ITERATION NO.: 14 OBJECTIVE VALUE: -14134.5077100038 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 111
PARAMETER: -2.2569E-01 -3.5945E-01 -4.2828E-01 -2.6031E-01 -1.2046E-01
GRADIENT: -1.4959E-01 1.3902E-01 7.6061E-02 1.5026E-01 8.2916E-02
0ITERATION NO.: 15 OBJECTIVE VALUE: -14134.5077558209 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 120
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
0ITERATION NO.: 16 OBJECTIVE VALUE: -14134.5077558209 NO.
OF FUNC. EVALS.: 6
CUMULATIVE NO. OF FUNC. EVALS.: 126
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
Elapsed estimation time in seconds: 8.85
#TERM:
0MINIMIZATION SUCCESSFUL
NO. OF FUNCTION EVALUATIONS USED: 126
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES,
AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS
0.
ETABAR: -3.6834E-05 -6.9519E-04
SE: 2.4522E-02 2.8113E-02
P VAL.: 9.9880E-01 9.8027E-01
ETAshrink(%): 9.8124E-02 4.3006E-01
EPSshrink(%): 1.7170E+00
Number of significant digits in final estimate.
Hi Kyun-Seop,
Thanks, this confirms what I was suspecting.
Naively I had hoped that it would give me the number of significant
digits in the reported values...
Unfortunately it seems that the number of significant digits can in the
reported values can be quite smaller then the "NO. OF SIG. DIGITS IN
FINAL EST."
>From the one experiment I've conducted since yesterday -comparing the
values of the .ext file produced by nonmem7.2 with PRINT 1 in $EST- I
got nsig=1.6 on the original scale and nsig=3.3 for the UCP.
So -as you seemed to have explored- used as a stop criterion not much
attention need to be paid to this value.
But as an assessment of the numerical precision of your estimates it can
be misleading.
Would this make a good summery?
K. Regads,
Xavier
Quoted reply history
From: Kyun-Seop Bae [mailto:[email protected]]
Sent: 10 October 2012 00:52
To: Woot de Trixhe, Xavier [JRDBE]; [email protected]
Subject: RE: [NMusers] Number of significant digits in final estimate.
Dear Xavier,
You can see the following comments at ZXMIN1.FOR within \NMVI\nm folder
(not for NONMEM 7):
C SIGD IS DISTANCE BETWEEN SOLUTION AND PUTATIVE MIN. PT.
C I.E., NUM OF SIG DIGITS SHARED BETWEEN THE TWO POINTS
"PUTATIVE MIN. PT" represents: current minimum point, current
iterated-estimate.
In a simple way, you can think it is the value like the following
SIG DIGITS = -log10(difference of current iterated-estimate and previous
iterated-estimate)
Sig digits exists for every theta, omega, sigma estimates, however
NONMEM prints out the minimum value among them.
(We use 10 as the base of logarithm, because we uses decimal numbers not
binary or octal numbers.)
(Usually printed iterated-estimates are not long (precise) enough to
calculate the above.)
This value is calculated with UCP(unconstrained parameter, previously it
is called as STP-scaled transformed parameter), not with original scale.
So you should not apply it to the original scale. Sig digits in original
scale is usually longer than the one printed.
If we say more theoretically, algorithmically some kind of boundary
(from current iterated-estimate to the true value) can be calculated
like the above equation:
Sig Digits (Digits of shared) = -log10(difference of current
iterated-estimate and true value)
Years ago, I thought this could supplement standard error (SE),
However, in my calculation, this boundary was usually larger than the
2*SE.
NONMEM uses this SIG DIGITS as one of the criteria for successful
termination, while other softwares do not.
So, my recommendation is that you should not give too much meaning or
importance to this.
While I test many covariate models searching for a full model, I do not
hesitate to use even SIG=1 option.
This may help you.
Kyun-Seop
=======================
Kyun-Seop Bae MD PhD
Associate Professor
Department of Clinical Pharmacology and Therapeutics
Asan Medical Center, University of Ulsan
88, Olympic-ro 43-gil, Sonpa-gu, Seoul 138-736, Republic of Korea
Email: [email protected]
Department Homepage: http://cpt.amc.seoul.kr
Blog: http://ksbae.blogspot.kr
From: [email protected] [mailto:[email protected]]
On Behalf Of Woot de Trixhe, Xavier [JRDBE]
Sent: Tuesday, October 09, 2012 9:00 PM
To: [email protected]
Subject: [NMusers] Number of significant digits in final estimate.
Hi,
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
I cannot seem to find any references to the method by which this number
is computed.
Xavier Woot de Trixhe, ir.
Scientist, AM&S
Clinical Pharmacology
Tel: +32 (0)14 60 29 70
Janssen Research & Development
Turnoudseweg 30
2340 Beerse, Belgium
0ITERATION NO.: 13 OBJECTIVE VALUE: -14134.5071734877 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 102
PARAMETER: -2.2522E-01 -3.6013E-01 -4.2825E-01 -2.6114E-01 -1.1966E-01
GRADIENT: 7.0164E-01 -5.8463E-01 6.5547E-01 -2.4262E-01 4.5721E-01
0ITERATION NO.: 14 OBJECTIVE VALUE: -14134.5077100038 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 111
PARAMETER: -2.2569E-01 -3.5945E-01 -4.2828E-01 -2.6031E-01 -1.2046E-01
GRADIENT: -1.4959E-01 1.3902E-01 7.6061E-02 1.5026E-01 8.2916E-02
0ITERATION NO.: 15 OBJECTIVE VALUE: -14134.5077558209 NO.
OF FUNC. EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 120
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
0ITERATION NO.: 16 OBJECTIVE VALUE: -14134.5077558209 NO.
OF FUNC. EVALS.: 6
CUMULATIVE NO. OF FUNC. EVALS.: 126
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
Elapsed estimation time in seconds: 8.85
#TERM:
0MINIMIZATION SUCCESSFUL
NO. OF FUNCTION EVALUATIONS USED: 126
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES,
AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS
0.
ETABAR: -3.6834E-05 -6.9519E-04
SE: 2.4522E-02 2.8113E-02
P VAL.: 9.9880E-01 9.8027E-01
ETAshrink(%): 9.8124E-02 4.3006E-01
EPSshrink(%): 1.7170E+00
Xavier,
NONMEM tries to minimize the objective function (i.e., -2 log likelihood). With
just one THETA, this function looks like a parabola. Given the data, the
location of the minimum is not uncertain - just as a calculated mean is not
uncertain - but here the minimum needs to be searched for via an iterative
process. The width of the parabola is associated with the uncertainty or
standard error of the location of the minimum. As I understand it, some
attention needs to be paid to the "NO. OF SIG. DIGITS" because it gives an
indication if the iterative process was behaving as desired. If not, the
location found, and the subsequent calculations, may perhaps not be trusted.
Erik
Quoted reply history
________________________________
From: [email protected] [[email protected]] on behalf of
Woot de Trixhe, Xavier [JRDBE] [[email protected]]
Sent: Wednesday, October 10, 2012 2:07 PM
To: [email protected]; [email protected]
Subject: RE: [NMusers] Number of significant digits in final estimate.
Hi Kyun-Seop,
Thanks, this confirms what I was suspecting.
Naively I had hoped that it would give me the number of significant digits in
the reported values…
Unfortunately it seems that the number of significant digits can in the
reported values can be quite smaller then the “NO. OF SIG. DIGITS IN FINAL EST.”
>From the one experiment I’ve conducted since yesterday -comparing the values
>of the .ext file produced by nonmem7.2 with PRINT 1 in $EST- I got nsig=1.6 on
>the original scale and nsig=3.3 for the UCP.
So –as you seemed to have explored– used as a stop criterion not much attention
need to be paid to this value.
But as an assessment of the numerical precision of your estimates it can be
misleading.
Would this make a good summery?
K. Regads,
Xavier
From: Kyun-Seop Bae [mailto:[email protected]]
Sent: 10 October 2012 00:52
To: Woot de Trixhe, Xavier [JRDBE]; [email protected]
Subject: RE: [NMusers] Number of significant digits in final estimate.
Dear Xavier,
You can see the following comments at ZXMIN1.FOR within \NMVI\nm folder (not
for NONMEM 7):
C SIGD IS DISTANCE BETWEEN SOLUTION AND PUTATIVE MIN. PT.
C I.E., NUM OF SIG DIGITS SHARED BETWEEN THE TWO POINTS
“PUTATIVE MIN. PT” represents: current minimum point, current iterated-estimate.
In a simple way, you can think it is the value like the following
SIG DIGITS = -log10(difference of current iterated-estimate and previous
iterated-estimate)
Sig digits exists for every theta, omega, sigma estimates, however NONMEM
prints out the minimum value among them.
(We use 10 as the base of logarithm, because we uses decimal numbers not binary
or octal numbers.)
(Usually printed iterated-estimates are not long (precise) enough to calculate
the above.)
This value is calculated with UCP(unconstrained parameter, previously it is
called as STP-scaled transformed parameter), not with original scale.
So you should not apply it to the original scale. Sig digits in original scale
is usually longer than the one printed.
If we say more theoretically, algorithmically some kind of boundary (from
current iterated-estimate to the true value) can be calculated like the above
equation:
Sig Digits (Digits of shared) = -log10(difference of current iterated-estimate
and true value)
Years ago, I thought this could supplement standard error (SE),
However, in my calculation, this boundary was usually larger than the 2*SE.
NONMEM uses this SIG DIGITS as one of the criteria for successful termination,
while other softwares do not.
So, my recommendation is that you should not give too much meaning or
importance to this.
While I test many covariate models searching for a full model, I do not
hesitate to use even SIG=1 option.
This may help you.
Kyun-Seop
=======================
Kyun-Seop Bae MD PhD
Associate Professor
Department of Clinical Pharmacology and Therapeutics
Asan Medical Center, University of Ulsan
88, Olympic-ro 43-gil, Sonpa-gu, Seoul 138-736, Republic of Korea
Email: [email protected]<mailto:[email protected]>
Department Homepage: http://cpt.amc.seoul.kr
Blog: http://ksbae.blogspot.kr
From: [email protected]<mailto:[email protected]>
[mailto:[email protected]] On Behalf Of Woot de Trixhe, Xavier
[JRDBE]
Sent: Tuesday, October 09, 2012 9:00 PM
To: [email protected]<mailto:[email protected]>
Subject: [NMusers] Number of significant digits in final estimate.
Hi,
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
I cannot seem to find any references to the method by which this number is
computed.
Xavier Woot de Trixhe, ir.
Scientist, AM&S
Clinical Pharmacology
Tel: +32 (0)14 60 29 70
Janssen Research & Development
Turnoudseweg 30
2340 Beerse, Belgium
0ITERATION NO.: 13 OBJECTIVE VALUE: -14134.5071734877 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 102
PARAMETER: -2.2522E-01 -3.6013E-01 -4.2825E-01 -2.6114E-01 -1.1966E-01
GRADIENT: 7.0164E-01 -5.8463E-01 6.5547E-01 -2.4262E-01 4.5721E-01
0ITERATION NO.: 14 OBJECTIVE VALUE: -14134.5077100038 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 111
PARAMETER: -2.2569E-01 -3.5945E-01 -4.2828E-01 -2.6031E-01 -1.2046E-01
GRADIENT: -1.4959E-01 1.3902E-01 7.6061E-02 1.5026E-01 8.2916E-02
0ITERATION NO.: 15 OBJECTIVE VALUE: -14134.5077558209 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 120
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
0ITERATION NO.: 16 OBJECTIVE VALUE: -14134.5077558209 NO. OF FUNC.
EVALS.: 6
CUMULATIVE NO. OF FUNC. EVALS.: 126
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
Elapsed estimation time in seconds: 8.85
#TERM:
0MINIMIZATION SUCCESSFUL
NO. OF FUNCTION EVALUATIONS USED: 126
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES,
AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0.
ETABAR: -3.6834E-05 -6.9519E-04
SE: 2.4522E-02 2.8113E-02
P VAL.: 9.9880E-01 9.8027E-01
ETAshrink(%): 9.8124E-02 4.3006E-01
EPSshrink(%): 1.7170E+00
Hi Xavier and Kyun-Seop,
I was hoping to have a more satisfactory answer as well, but it's good to have
the real answer.
I have always looked at nsig as a general marker of stability while I think of
the actual number of significant digits in the final answer as related to the
RSE of the parameter. Typically, I report the point estimate and a CI; the
number of significant digits I report is typically 3, and I allow for the CI to
provide interpretation of how certain I am of the result.
Thanks,
Bill
Quoted reply history
From: [email protected] [mailto:[email protected]] On
Behalf Of Woot de Trixhe, Xavier [JRDBE]
Sent: Wednesday, October 10, 2012 8:08 AM
To: [email protected]; [email protected]
Subject: RE: [NMusers] Number of significant digits in final estimate.
Hi Kyun-Seop,
Thanks, this confirms what I was suspecting.
Naively I had hoped that it would give me the number of significant digits in
the reported values...
Unfortunately it seems that the number of significant digits can in the
reported values can be quite smaller then the "NO. OF SIG. DIGITS IN FINAL EST."
>From the one experiment I've conducted since yesterday -comparing the values
>of the .ext file produced by nonmem7.2 with PRINT 1 in $EST- I got nsig=1.6 on
>the original scale and nsig=3.3 for the UCP.
So -as you seemed to have explored- used as a stop criterion not much attention
need to be paid to this value.
But as an assessment of the numerical precision of your estimates it can be
misleading.
Would this make a good summery?
K. Regads,
Xavier
From: Kyun-Seop Bae [mailto:[email protected]]
Sent: 10 October 2012 00:52
To: Woot de Trixhe, Xavier [JRDBE]; [email protected]
Subject: RE: [NMusers] Number of significant digits in final estimate.
Dear Xavier,
You can see the following comments at ZXMIN1.FOR within \NMVI\nm folder (not
for NONMEM 7):
C SIGD IS DISTANCE BETWEEN SOLUTION AND PUTATIVE MIN. PT.
C I.E., NUM OF SIG DIGITS SHARED BETWEEN THE TWO POINTS
"PUTATIVE MIN. PT" represents: current minimum point, current iterated-estimate.
In a simple way, you can think it is the value like the following
SIG DIGITS = -log10(difference of current iterated-estimate and previous
iterated-estimate)
Sig digits exists for every theta, omega, sigma estimates, however NONMEM
prints out the minimum value among them.
(We use 10 as the base of logarithm, because we uses decimal numbers not binary
or octal numbers.)
(Usually printed iterated-estimates are not long (precise) enough to calculate
the above.)
This value is calculated with UCP(unconstrained parameter, previously it is
called as STP-scaled transformed parameter), not with original scale.
So you should not apply it to the original scale. Sig digits in original scale
is usually longer than the one printed.
If we say more theoretically, algorithmically some kind of boundary (from
current iterated-estimate to the true value) can be calculated like the above
equation:
Sig Digits (Digits of shared) = -log10(difference of current iterated-estimate
and true value)
Years ago, I thought this could supplement standard error (SE),
However, in my calculation, this boundary was usually larger than the 2*SE.
NONMEM uses this SIG DIGITS as one of the criteria for successful termination,
while other softwares do not.
So, my recommendation is that you should not give too much meaning or
importance to this.
While I test many covariate models searching for a full model, I do not
hesitate to use even SIG=1 option.
This may help you.
Kyun-Seop
=======================
Kyun-Seop Bae MD PhD
Associate Professor
Department of Clinical Pharmacology and Therapeutics
Asan Medical Center, University of Ulsan
88, Olympic-ro 43-gil, Sonpa-gu, Seoul 138-736, Republic of Korea
Email: [email protected]<mailto:[email protected]>
Department Homepage: http://cpt.amc.seoul.kr
Blog: http://ksbae.blogspot.kr
From: [email protected]<mailto:[email protected]>
[mailto:[email protected]] On Behalf Of Woot de Trixhe, Xavier
[JRDBE]
Sent: Tuesday, October 09, 2012 9:00 PM
To: [email protected]<mailto:[email protected]>
Subject: [NMusers] Number of significant digits in final estimate.
Hi,
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
I cannot seem to find any references to the method by which this number is
computed.
Xavier Woot de Trixhe, ir.
Scientist, AM&S
Clinical Pharmacology
Tel: +32 (0)14 60 29 70
Janssen Research & Development
Turnoudseweg 30
2340 Beerse, Belgium
0ITERATION NO.: 13 OBJECTIVE VALUE: -14134.5071734877 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 102
PARAMETER: -2.2522E-01 -3.6013E-01 -4.2825E-01 -2.6114E-01 -1.1966E-01
GRADIENT: 7.0164E-01 -5.8463E-01 6.5547E-01 -2.4262E-01 4.5721E-01
0ITERATION NO.: 14 OBJECTIVE VALUE: -14134.5077100038 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 111
PARAMETER: -2.2569E-01 -3.5945E-01 -4.2828E-01 -2.6031E-01 -1.2046E-01
GRADIENT: -1.4959E-01 1.3902E-01 7.6061E-02 1.5026E-01 8.2916E-02
0ITERATION NO.: 15 OBJECTIVE VALUE: -14134.5077558209 NO. OF FUNC.
EVALS.: 9
CUMULATIVE NO. OF FUNC. EVALS.: 120
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
0ITERATION NO.: 16 OBJECTIVE VALUE: -14134.5077558209 NO. OF FUNC.
EVALS.: 6
CUMULATIVE NO. OF FUNC. EVALS.: 126
PARAMETER: -2.2560E-01 -3.5959E-01 -4.2829E-01 -2.6071E-01 -1.2073E-01
GRADIENT: 8.5892E-03 -2.6632E-03 -8.2743E-02 -3.7853E-02 -4.3828E-02
Elapsed estimation time in seconds: 8.85
#TERM:
0MINIMIZATION SUCCESSFUL
NO. OF FUNCTION EVALUATIONS USED: 126
NO. OF SIG. DIGITS IN FINAL EST.: 3.1
ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES,
AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0.
ETABAR: -3.6834E-05 -6.9519E-04
SE: 2.4522E-02 2.8113E-02
P VAL.: 9.9880E-01 9.8027E-01
ETAshrink(%): 9.8124E-02 4.3006E-01
EPSshrink(%): 1.7170E+00