Dear all,
Does anyone witnessed such a phenomenon in NONMEM as when you reduced an
ETA, the OFV value, rather than increase, actually decreased? It's quite
against intuition, as individual estimation should be better than
population estimation in that particular parameter. Both models, whether
having this ETA, converged very well.
Best
--
Xinting
Reducing ETAs actually decreased OFV
You mean you removed an eta and the objective function went down? I dont think
this can really happen in a straightforward way.
In NONMEM, the minimum of the objective function is found and if having all
etas to zero gives a lower objective function than some other eta values then
barring covergence problems all zero etas should have been found.
warm regards,
Douglas Eleveld
Quoted reply history
________________________________________
From: [email protected] [[email protected]] on behalf of
Xinting Wang [[email protected]]
Sent: Sunday, August 11, 2013 4:23 AM
To: [email protected]
Subject: [NMusers] Reducing ETAs actually decreased OFV
Dear all,
Does anyone witnessed such a phenomenon in NONMEM as when you reduced an ETA,
the OFV value, rather than increase, actually decreased? It's quite against
intuition, as individual estimation should be better than population estimation
in that particular parameter. Both models, whether having this ETA, converged
very well.
Best
--
Xinting
________________________________
Xinting,
Try to start from the initial conditions of your "reduced" model but add that "reduced" ETA with the corresponding OMEGA equal to 0.01 or other small number. If the control stream code is correct, the objective function should decrease or retain the same value.
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 8/10/2013 10:23 PM, Xinting Wang wrote:
> Dear all,
>
> Does anyone witnessed such a phenomenon in NONMEM as when you reduced an
> ETA, the OFV value, rather than increase, actually decreased? It's quite
> against intuition, as individual estimation should be better than
> population estimation in that particular parameter. Both models, whether
> having this ETA, converged very well.
>
> Best
>
> --
> Xinting
Hi Xinting,
In a few rare cases, I've seen this happen if the model is approaching
nonconvergence. In those cases, typically the RSE on one or more parameters
will increase and the ratio of max to min eigenvalues will increase
substantially. Are you seeing either of these?
Thanks,
Bill
Quoted reply history
On Aug 11, 2013, at 21:56, "Leonid Gibiansky" <[email protected]> wrote:
Xinting,
Try to start from the initial conditions of your "reduced" model but add that
"reduced" ETA with the corresponding OMEGA equal to 0.01 or other small number.
If the control stream code is correct, the objective function should decrease
or retain the same value.
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
On 8/10/2013 10:23 PM, Xinting Wang wrote:
> Dear all,
>
> Does anyone witnessed such a phenomenon in NONMEM as when you reduced an
> ETA, the OFV value, rather than increase, actually decreased? It's quite
> against intuition, as individual estimation should be better than
> population estimation in that particular parameter. Both models, whether
> having this ETA, converged very well.
>
> Best
>
> --
> Xinting
Dear Leonid,
I tried with your method and found the same result. The initial estimation
of the added ETA was set at 0.01, and the result showed an increase of OFV.
Please see below the $PK part of the control file for more information.
Many thanks.
Dear Bill,
Could you please explain that in a little bit more detail? I am pasting the
$PK part of the control file in case you could find the useful information.
Thanks a lot.
$PK
FA1=0
FA2=0
FA3=0
FA4=0
IF(DOSE.EQ.250) THEN
FA1=1
ENDIF
IF(DOSE.EQ.500) THEN
FA2=1
ENDIF
IF(DOSE.EQ.850) THEN
FA3=1
ENDIF
IF(DOSE.EQ.1000) THEN
FA4=1
ENDIF
F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
TVCL=THETA(1)
TVV2=THETA(2)
TVKA=THETA(3)
TVQ=THETA(4)
TVV3=THETA(5)
CL=TVCL*EXP(ETA(1))
V2=TVV2*EXP(ETA(2))
KA=TVKA*EXP(ETA(5))
Q=TVQ*EXP(ETA(3))
V3=TVV3*EXP(ETA(4))
S2=V2/1000
S3=V3/1000
$ERROR
IPRE=F
IRES=DV-IPRE
W=F
IF(W.EQ.0) W = 1
IWRE = IRES/W
Y=F*(1+EPS(1))+EPS(2)
Best Regards
Quoted reply history
On 12 August 2013 20:50, Denney, William S. <[email protected]>wrote:
> Hi Xinting,
>
> In a few rare cases, I've seen this happen if the model is approaching
> nonconvergence. In those cases, typically the RSE on one or more
> parameters will increase and the ratio of max to min eigenvalues will
> increase substantially. Are you seeing either of these?
>
> Thanks,
>
> Bill
>
> On Aug 11, 2013, at 21:56, "Leonid Gibiansky" <[email protected]>
> wrote:
>
> Xinting,
> Try to start from the initial conditions of your "reduced" model but add
> that "reduced" ETA with the corresponding OMEGA equal to 0.01 or other
> small number. If the control stream code is correct, the objective function
> should decrease or retain the same value.
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com
> tel: (301) 767 5566
>
>
>
> On 8/10/2013 10:23 PM, Xinting Wang wrote:
> > Dear all,
> >
> > Does anyone witnessed such a phenomenon in NONMEM as when you reduced an
> > ETA, the OFV value, rather than increase, actually decreased? It's quite
> > against intuition, as individual estimation should be better than
> > population estimation in that particular parameter. Both models, whether
> > having this ETA, converged very well.
> >
> > Best
> >
> > --
> > Xinting
>
--
Xinting
Hi Xinting,
When I've seen this before, it is due to the model approaching non-convergence.
Given the model that you're showing below, I'd guess that your issue is coming
from the eta on Q or V3 because it is rare to have enough data to fit true IIV
there.
Thanks,
Bill
Quoted reply history
On Aug 25, 2013, at 8:42, "Xinting Wang"
<[email protected]<mailto:[email protected]>> wrote:
Dear Leonid,
I tried with your method and found the same result. The initial estimation of
the added ETA was set at 0.01, and the result showed an increase of OFV. Please
see below the $PK part of the control file for more information. Many thanks.
Dear Bill,
Could you please explain that in a little bit more detail? I am pasting the $PK
part of the control file in case you could find the useful information. Thanks
a lot.
$PK
FA1=0
FA2=0
FA3=0
FA4=0
IF(DOSE.EQ.250) THEN
FA1=1
ENDIF
IF(DOSE.EQ.500) THEN
FA2=1
ENDIF
IF(DOSE.EQ.850) THEN
FA3=1
ENDIF
IF(DOSE.EQ.1000) THEN
FA4=1
ENDIF
F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
TVCL=THETA(1)
TVV2=THETA(2)
TVKA=THETA(3)
TVQ=THETA(4)
TVV3=THETA(5)
CL=TVCL*EXP(ETA(1))
V2=TVV2*EXP(ETA(2))
KA=TVKA*EXP(ETA(5))
Q=TVQ*EXP(ETA(3))
V3=TVV3*EXP(ETA(4))
S2=V2/1000
S3=V3/1000
$ERROR
IPRE=F
IRES=DV-IPRE
W=F
IF(W.EQ.0) W = 1
IWRE = IRES/W
Y=F*(1+EPS(1))+EPS(2)
Best Regards
On 12 August 2013 20:50, Denney, William S.
<[email protected]<mailto:[email protected]>> wrote:
Hi Xinting,
In a few rare cases, I've seen this happen if the model is approaching
nonconvergence. In those cases, typically the RSE on one or more parameters
will increase and the ratio of max to min eigenvalues will increase
substantially. Are you seeing either of these?
Thanks,
Bill
On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
<[email protected]<mailto:[email protected]>> wrote:
Xinting,
Try to start from the initial conditions of your "reduced" model but add that
"reduced" ETA with the corresponding OMEGA equal to 0.01 or other small number.
If the control stream code is correct, the objective function should decrease
or retain the same value.
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: http://www.quantpharm.com
e-mail: LGibiansky at http://quantpharm.com
tel: (301) 767 5566<tel:%28301%29%20767%205566>
On 8/10/2013 10:23 PM, Xinting Wang wrote:
> Dear all,
>
> Does anyone witnessed such a phenomenon in NONMEM as when you reduced an
> ETA, the OFV value, rather than increase, actually decreased? It's quite
> against intuition, as individual estimation should be better than
> population estimation in that particular parameter. Both models, whether
> having this ETA, converged very well.
>
> Best
>
> --
> Xinting
--
Xinting
Hi Xinting,
You should be able to do it. Let's check it again this way
1. You run the model with all ETAs included, but one ETA (the one that was excluded in the reduced model) is fixed to zero. You should be able to reproduce your "reduced ETA" result (OF) 2. You take the same control stream, and set all initial values to the final parameter estimates of model (1) above, except you use the small value (may be not 0.01 but 0.000001) as the initial value of the ETA that was fixed to zero in model (1).
Model (2) is the not-reduced model, and it's OF should be less or equal to the OF of model (1). If this is not the case, increase the number of significant digits in the initial estimates of model (2) - take those from the final estimates of model 1.
Without data, it is very difficult to offer more specific advice.
Also, what is the magnitude of the OF change? What is the estimate of the OMEGA for the ETA in question?
Regards,
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 8/25/2013 8:42 AM, Xinting Wang wrote:
> Dear Leonid,
>
> I tried with your method and found the same result. The initial
> estimation of the added ETA was set at 0.01, and the result showed an
> increase of OFV. Please see below the $PK part of the control file for
> more information. Many thanks.
>
> Dear Bill,
>
> Could you please explain that in a little bit more detail? I am pasting
> the $PK part of the control file in case you could find the useful
> information. Thanks a lot.
>
> $PK
>
> FA1=0
> FA2=0
> FA3=0
> FA4=0
>
> IF(DOSE.EQ.250) THEN
> FA1=1
> ENDIF
>
> IF(DOSE.EQ.500) THEN
> FA2=1
> ENDIF
>
> IF(DOSE.EQ.850) THEN
> FA3=1
> ENDIF
>
> IF(DOSE.EQ.1000) THEN
> FA4=1
> ENDIF
>
> F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
>
> TVCL=THETA(1)
> TVV2=THETA(2)
> TVKA=THETA(3)
> TVQ=THETA(4)
> TVV3=THETA(5)
>
> CL=TVCL*EXP(ETA(1))
> V2=TVV2*EXP(ETA(2))
> KA=TVKA*EXP(ETA(5))
> Q=TVQ*EXP(ETA(3))
> V3=TVV3*EXP(ETA(4))
>
> S2=V2/1000
> S3=V3/1000
>
> $ERROR
>
> IPRE=F
>
> IRES=DV-IPRE
>
> W=F
>
> IF(W.EQ.0) W = 1
>
> IWRE = IRES/W
>
> Y=F*(1+EPS(1))+EPS(2)
>
> Best Regards
>
> On 12 August 2013 20:50, Denney, William S. <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Xinting,
>
> In a few rare cases, I've seen this happen if the model is
> approaching nonconvergence. In those cases, typically the RSE on
> one or more parameters will increase and the ratio of max to min
> eigenvalues will increase substantially. Are you seeing either of
> these?
>
> Thanks,
>
> Bill
>
> On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
> <[email protected] <mailto:[email protected]>> wrote:
>
> Xinting,
> Try to start from the initial conditions of your "reduced" model but
> add that "reduced" ETA with the corresponding OMEGA equal to 0.01 or
> other small number. If the control stream code is correct, the
> objective function should decrease or retain the same value.
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com http://www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
>
> On 8/10/2013 10:23 PM, Xinting Wang wrote:
> > Dear all,
> >
> > Does anyone witnessed such a phenomenon in NONMEM as when you
> reduced an
> > ETA, the OFV value, rather than increase, actually decreased?
> It's quite
> > against intuition, as individual estimation should be better than
> > population estimation in that particular parameter. Both models,
> whether
> > having this ETA, converged very well.
> >
> > Best
> >
> > --
> > Xinting
>
> --
> Xinting
Dear Bill,
Appreciate your reply a lot. The issue is from KA. Adding KA or not did
have this problem. However, regarding your statement "it is rare to have
enough data to fit true IIV", can you explain more about this. My data set
is from Phase I studies, and I thought this should be enough for this
simulation.
Dear Leonid,
Thanks very much for your detailed suggestion. I followed the steps you
listed above, and did find that the OFV decreased in step 2, just as you
predicted. Then using the estimation to replace the initial values for all
of the THETA, OMEGA and SIGMA, the OFV stabilized. However, I am curious
about the explanation for this. And, is this a universal method for
estimation of initial values? Thank you.
The change of OFV was around 100 (the OFV was ~114300). I am pasting the
OMEGA matrix below for your information.
ETA1 ETA2 ETA3 ETA4 ETA5
ETA1
+ 4.08E-02
ETA2
+ 0.00E+00 1.57E-01
ETA3
+ 0.00E+00 0.00E+00 1.30E-01
ETA4
+ 0.00E+00 0.00E+00 0.00E+00 4.07E-01
ETA5
+ 0.00E+00 0.00E+00 0.00E+00 0.00E+00 2.19E-02
ETA5 (0.0219) is the one caused the problem.
Best Regards
Quoted reply history
On 26 August 2013 07:06, Leonid Gibiansky <[email protected]> wrote:
> Hi Xinting,
> You should be able to do it. Let's check it again this way
> 1. You run the model with all ETAs included, but one ETA (the one that was
> excluded in the reduced model) is fixed to zero. You should be able to
> reproduce your "reduced ETA" result (OF)
> 2. You take the same control stream, and set all initial values to the
> final parameter estimates of model (1) above, except you use the small
> value (may be not 0.01 but 0.000001) as the initial value of the ETA that
> was fixed to zero in model (1).
>
> Model (2) is the not-reduced model, and it's OF should be less or equal to
> the OF of model (1). If this is not the case, increase the number of
> significant digits in the initial estimates of model (2) - take those from
> the final estimates of model 1.
>
> Without data, it is very difficult to offer more specific advice.
>
> Also, what is the magnitude of the OF change? What is the estimate of the
> OMEGA for the ETA in question?
>
> Regards,
>
> Leonid
>
>
>
>
> ------------------------------**--------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com
> tel: (301) 767 5566
>
>
>
> On 8/25/2013 8:42 AM, Xinting Wang wrote:
>
>> Dear Leonid,
>>
>> I tried with your method and found the same result. The initial
>> estimation of the added ETA was set at 0.01, and the result showed an
>> increase of OFV. Please see below the $PK part of the control file for
>> more information. Many thanks.
>>
>> Dear Bill,
>>
>> Could you please explain that in a little bit more detail? I am pasting
>> the $PK part of the control file in case you could find the useful
>> information. Thanks a lot.
>>
>> $PK
>>
>> FA1=0
>> FA2=0
>> FA3=0
>> FA4=0
>>
>> IF(DOSE.EQ.250) THEN
>> FA1=1
>> ENDIF
>>
>> IF(DOSE.EQ.500) THEN
>> FA2=1
>> ENDIF
>>
>> IF(DOSE.EQ.850) THEN
>> FA3=1
>> ENDIF
>>
>> IF(DOSE.EQ.1000) THEN
>> FA4=1
>> ENDIF
>>
>> F1=FA1+FA2*THETA(6)+FA3*THETA(**7)+FA4*THETA(8)
>>
>> TVCL=THETA(1)
>> TVV2=THETA(2)
>> TVKA=THETA(3)
>> TVQ=THETA(4)
>> TVV3=THETA(5)
>>
>> CL=TVCL*EXP(ETA(1))
>> V2=TVV2*EXP(ETA(2))
>> KA=TVKA*EXP(ETA(5))
>> Q=TVQ*EXP(ETA(3))
>> V3=TVV3*EXP(ETA(4))
>>
>>
>> S2=V2/1000
>> S3=V3/1000
>>
>>
>> $ERROR
>>
>> IPRE=F
>>
>> IRES=DV-IPRE
>>
>> W=F
>>
>> IF(W.EQ.0) W = 1
>>
>> IWRE = IRES/W
>>
>> Y=F*(1+EPS(1))+EPS(2)
>>
>> Best Regards
>>
>>
>> On 12 August 2013 20:50, Denney, William S. <[email protected]
>> <mailto:William.S.Denney@**pfizer.com <[email protected]>>>
>> wrote:
>>
>> Hi Xinting,
>>
>> In a few rare cases, I've seen this happen if the model is
>> approaching nonconvergence. In those cases, typically the RSE on
>> one or more parameters will increase and the ratio of max to min
>> eigenvalues will increase substantially. Are you seeing either of
>> these?
>>
>> Thanks,
>>
>> Bill
>>
>> On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
>> <[email protected]
>> <mailto:lgibiansky@quantpharm.**com<[email protected]>>>
>> wrote:
>>
>> Xinting,
>> Try to start from the initial conditions of your "reduced" model but
>> add that "reduced" ETA with the corresponding OMEGA equal to 0.01 or
>> other small number. If the control stream code is correct, the
>> objective function should decrease or retain the same value.
>> Leonid
>>
>> ------------------------------**--------
>> Leonid Gibiansky, Ph.D.
>> President, QuantPharm LLC
>> web: www.quantpharm.com http://www.quantpharm.com
>>
>> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
>> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
>>
>>
>>
>> On 8/10/2013 10:23 PM, Xinting Wang wrote:
>> > Dear all,
>> >
>> > Does anyone witnessed such a phenomenon in NONMEM as when you
>> reduced an
>> > ETA, the OFV value, rather than increase, actually decreased?
>> It's quite
>> > against intuition, as individual estimation should be better than
>> > population estimation in that particular parameter. Both models,
>> whether
>> > having this ETA, converged very well.
>> >
>> > Best
>> >
>> > --
>> > Xinting
>>
>>
>>
>>
>> --
>> Xinting
>>
>
--
Xinting
Hi Xinting,
This is a rather broad (and often highly-opinionated) topic. At the highest
level, you can only fit parameters in a model where you have enough data to
estimate the parameter. A simple example is that if you have data that you
want to fit an Emax model to with measurements only up to the EC10, you don't
have enough data to estimate Emax and ED50; it will look fully linear.
Due to data variability and the fact that Q and V3 are less correlated with the
measurements than other parameters (Ka, V2, and CL have a stronger effect on
the measurements than Q and V3), the estimates will be more difficult. A good
example of how to evaluate this would be what Peter Bonate just suggested: do
likelihood profiling on each of the parameters (especially the ETAs) to
estimate the certainty (peakedness) or uncertainty (flatness) in the parameter
estimates.
Thanks,
Bill
Quoted reply history
From: Xinting Wang [mailto:[email protected]]
Sent: Monday, August 26, 2013 9:27 AM
To: Leonid Gibiansky
Cc: [email protected]; Denney, William S.
Subject: Re: [NMusers] Reducing ETAs actually decreased OFV
Dear Bill,
Appreciate your reply a lot. The issue is from KA. Adding KA or not did have
this problem. However, regarding your statement "it is rare to have enough data
to fit true IIV", can you explain more about this. My data set is from Phase I
studies, and I thought this should be enough for this simulation.
Dear Leonid,
Thanks very much for your detailed suggestion. I followed the steps you listed
above, and did find that the OFV decreased in step 2, just as you predicted.
Then using the estimation to replace the initial values for all of the THETA,
OMEGA and SIGMA, the OFV stabilized. However, I am curious about the
explanation for this. And, is this a universal method for estimation of initial
values? Thank you.
The change of OFV was around 100 (the OFV was ~114300). I am pasting the OMEGA
matrix below for your information.
ETA1 ETA2 ETA3 ETA4 ETA5
ETA1
+ 4.08E-02
ETA2
+ 0.00E+00 1.57E-01
ETA3
+ 0.00E+00 0.00E+00 1.30E-01
ETA4
+ 0.00E+00 0.00E+00 0.00E+00 4.07E-01
ETA5
+ 0.00E+00 0.00E+00 0.00E+00 0.00E+00 2.19E-02
ETA5 (0.0219) is the one caused the problem.
Best Regards
On 26 August 2013 07:06, Leonid Gibiansky
<[email protected]<mailto:[email protected]>> wrote:
Hi Xinting,
You should be able to do it. Let's check it again this way
1. You run the model with all ETAs included, but one ETA (the one that was
excluded in the reduced model) is fixed to zero. You should be able to
reproduce your "reduced ETA" result (OF)
2. You take the same control stream, and set all initial values to the final
parameter estimates of model (1) above, except you use the small value (may be
not 0.01 but 0.000001) as the initial value of the ETA that was fixed to zero
in model (1).
Model (2) is the not-reduced model, and it's OF should be less or equal to the
OF of model (1). If this is not the case, increase the number of significant
digits in the initial estimates of model (2) - take those from the final
estimates of model 1.
Without data, it is very difficult to offer more specific advice.
Also, what is the magnitude of the OF change? What is the estimate of the OMEGA
for the ETA in question?
Regards,
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: http://www.quantpharm.com
e-mail: LGibiansky at http://quantpharm.com
tel: (301) 767 5566<tel:%28301%29%20767%205566>
On 8/25/2013 8:42 AM, Xinting Wang wrote:
Dear Leonid,
I tried with your method and found the same result. The initial
estimation of the added ETA was set at 0.01, and the result showed an
increase of OFV. Please see below the $PK part of the control file for
more information. Many thanks.
Dear Bill,
Could you please explain that in a little bit more detail? I am pasting
the $PK part of the control file in case you could find the useful
information. Thanks a lot.
$PK
FA1=0
FA2=0
FA3=0
FA4=0
IF(DOSE.EQ.250) THEN
FA1=1
ENDIF
IF(DOSE.EQ.500) THEN
FA2=1
ENDIF
IF(DOSE.EQ.850) THEN
FA3=1
ENDIF
IF(DOSE.EQ.1000) THEN
FA4=1
ENDIF
F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
TVCL=THETA(1)
TVV2=THETA(2)
TVKA=THETA(3)
TVQ=THETA(4)
TVV3=THETA(5)
CL=TVCL*EXP(ETA(1))
V2=TVV2*EXP(ETA(2))
KA=TVKA*EXP(ETA(5))
Q=TVQ*EXP(ETA(3))
V3=TVV3*EXP(ETA(4))
S2=V2/1000
S3=V3/1000
$ERROR
IPRE=F
IRES=DV-IPRE
W=F
IF(W.EQ.0) W = 1
IWRE = IRES/W
Y=F*(1+EPS(1))+EPS(2)
Best Regards
On 12 August 2013 20:50, Denney, William S.
<[email protected]<mailto:[email protected]>
<mailto:[email protected]<mailto:[email protected]>>> wrote:
Hi Xinting,
In a few rare cases, I've seen this happen if the model is
approaching nonconvergence. In those cases, typically the RSE on
one or more parameters will increase and the ratio of max to min
eigenvalues will increase substantially. Are you seeing either of
these?
Thanks,
Bill
On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
<[email protected]<mailto:[email protected]>
<mailto:[email protected]<mailto:[email protected]>>> wrote:
Xinting,
Try to start from the initial conditions of your "reduced" model but
add that "reduced" ETA with the corresponding OMEGA equal to 0.01 or
other small number. If the control stream code is correct, the
objective function should decrease or retain the same value.
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: http://www.quantpharm.com
http://www.quantpharm.com
e-mail: LGibiansky at http://quantpharm.com
http://quantpharm.com
tel: (301) 767 5566<tel:%28301%29%20767%205566> <tel:%28301%29%20767%205566>
On 8/10/2013 10:23 PM, Xinting Wang wrote:
> Dear all,
>
> Does anyone witnessed such a phenomenon in NONMEM as when you
reduced an
> ETA, the OFV value, rather than increase, actually decreased?
It's quite
> against intuition, as individual estimation should be better than
> population estimation in that particular parameter. Both models,
whether
> having this ETA, converged very well.
>
> Best
>
> --
> Xinting
--
Xinting
--
Xinting
I am not sure that you need likelihood profiling or any other sophisticated procedures to study this particular problem. You can look at relative standard errors of the parameter estimates: if one of the ETAs is poorly estimated, this is the candidate for removal. For two-compartment models, it is rarely possible to estimate ETAs on peripheral compartment, and at least one of those can be removed (usually).
If the goal is to describe the data, you look for the simplest model that allow you to fit the data. You may start with the model with all random effects, but then try to reduce the number of random effect (unless you use new IMP/SAEM/BAYES type procedures) to arrive at the simpler model. You may use OF as a guide: if OF drop is small when you remove the ETA, this ETA does not contribute to the fit (and the model can equally well fit the data without this particular ETA). Alternative procedure is to compare full (with ETAs) and reduced (with one ETA fixed to zero) model using various diagnostic plots procedure (VPC in particular), or plots of one model versus the other model: PRED vs PRED and IPRED vs IPRED (where PRED and IPRED belog to two models that you are comparing). If these plots looks like identity lines (both in normal and log axes), you can safely use simpler model, especially if VPC results are similar or identical.
As to the specific procedure that allowed you to fix the strange OF behavior, even the simple problems (like two-compartment model that was used) are highly nonlinear, and gradient methods cannot guarantee the global minimum. The solution (local minimum) may depend on initial conditions. By starting from the solution of the reduced problem, you put the model in the vicinity of the correct local minimum, while when you started from the larger model, it converged to the different minimum. This is not a universal procedure, but it helps time to time if the model has difficulties finding the solution.
Regards,
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 8/26/2013 9:58 AM, Denney, William S. wrote:
> Hi Xinting,
>
> This is a rather broad (and often highly-opinionated) topic. At the
> highest level, you can only fit parameters in a model where you have
> enough data to estimate the parameter. A simple example is that if you
> have data that you want to fit an Emax model to with measurements only
> up to the EC10, you don’t have enough data to estimate Emax and ED50; it
> will look fully linear.
>
> Due to data variability and the fact that Q and V3 are less correlated
> with the measurements than other parameters (Ka, V2, and CL have a
> stronger effect on the measurements than Q and V3), the estimates will
> be more difficult. A good example of how to evaluate this would be what
> Peter Bonate just suggested: do likelihood profiling on each of the
> parameters (especially the ETAs) to estimate the certainty (peakedness)
> or uncertainty (flatness) in the parameter estimates.
>
> Thanks,
>
> Bill
>
> *From:*Xinting Wang [mailto:[email protected]]
> *Sent:* Monday, August 26, 2013 9:27 AM
> *To:* Leonid Gibiansky
> *Cc:* [email protected]; Denney, William S.
> *Subject:* Re: [NMusers] Reducing ETAs actually decreased OFV
>
> Dear Bill,
>
> Appreciate your reply a lot. The issue is from KA. Adding KA or not did
> have this problem. However, regarding your statement "it is rare to have
> enough data to fit true IIV", can you explain more about this. My data
> set is from Phase I studies, and I thought this should be enough for
> this simulation.
>
> Dear Leonid,
>
> Thanks very much for your detailed suggestion. I followed the steps you
> listed above, and did find that the OFV decreased in step 2, just as you
> predicted. Then using the estimation to replace the initial values for
> all of the THETA, OMEGA and SIGMA, the OFV stabilized. However, I am
> curious about the explanation for this. And, is this a universal method
> for estimation of initial values? Thank you.
>
> The change of OFV was around 100 (the OFV was ~114300). I am pasting the
> OMEGA matrix below for your information.
>
> ETA1 ETA2 ETA3 ETA4 ETA5
>
> ETA1
> + 4.08E-02
>
> ETA2
> + 0.00E+00 1.57E-01
>
> ETA3
> + 0.00E+00 0.00E+00 1.30E-01
>
> ETA4
> + 0.00E+00 0.00E+00 0.00E+00 4.07E-01
>
> ETA5
> + 0.00E+00 0.00E+00 0.00E+00 0.00E+00 2.19E-02
>
> ETA5 (0.0219) is the one caused the problem.
>
> Best Regards
>
> On 26 August 2013 07:06, Leonid Gibiansky <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Xinting,
> You should be able to do it. Let's check it again this way
> 1. You run the model with all ETAs included, but one ETA (the one that
> was excluded in the reduced model) is fixed to zero. You should be able
> to reproduce your "reduced ETA" result (OF)
> 2. You take the same control stream, and set all initial values to the
> final parameter estimates of model (1) above, except you use the small
> value (may be not 0.01 but 0.000001) as the initial value of the ETA
> that was fixed to zero in model (1).
>
> Model (2) is the not-reduced model, and it's OF should be less or equal
> to the OF of model (1). If this is not the case, increase the number of
> significant digits in the initial estimates of model (2) - take those
> from the final estimates of model 1.
>
> Without data, it is very difficult to offer more specific advice.
>
> Also, what is the magnitude of the OF change? What is the estimate of
> the OMEGA for the ETA in question?
>
> Regards,
>
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com http://www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
>
> On 8/25/2013 8:42 AM, Xinting Wang wrote:
>
> Dear Leonid,
>
> I tried with your method and found the same result. The initial
> estimation of the added ETA was set at 0.01, and the result showed an
> increase of OFV. Please see below the $PK part of the control file for
> more information. Many thanks.
>
> Dear Bill,
>
> Could you please explain that in a little bit more detail? I am pasting
> the $PK part of the control file in case you could find the useful
> information. Thanks a lot.
>
> $PK
>
> FA1=0
> FA2=0
> FA3=0
> FA4=0
>
> IF(DOSE.EQ.250) THEN
> FA1=1
> ENDIF
>
> IF(DOSE.EQ.500) THEN
> FA2=1
> ENDIF
>
> IF(DOSE.EQ.850) THEN
> FA3=1
> ENDIF
>
> IF(DOSE.EQ.1000) THEN
> FA4=1
> ENDIF
>
> F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
>
> TVCL=THETA(1)
> TVV2=THETA(2)
> TVKA=THETA(3)
> TVQ=THETA(4)
> TVV3=THETA(5)
>
> CL=TVCL*EXP(ETA(1))
> V2=TVV2*EXP(ETA(2))
> KA=TVKA*EXP(ETA(5))
> Q=TVQ*EXP(ETA(3))
> V3=TVV3*EXP(ETA(4))
>
> S2=V2/1000
> S3=V3/1000
>
> $ERROR
>
> IPRE=F
>
> IRES=DV-IPRE
>
> W=F
>
> IF(W.EQ.0) W = 1
>
> IWRE = IRES/W
>
> Y=F*(1+EPS(1))+EPS(2)
>
> Best Regards
>
> On 12 August 2013 20:50, Denney, William S.
> <[email protected] <mailto:[email protected]>
>
> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
>
> Hi Xinting,
>
> In a few rare cases, I've seen this happen if the model is
> approaching nonconvergence. In those cases, typically the RSE on
> one or more parameters will increase and the ratio of max to min
> eigenvalues will increase substantially. Are you seeing either of
> these?
>
> Thanks,
>
> Bill
>
> On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
>
> <[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
>
> Xinting,
> Try to start from the initial conditions of your "reduced"
> model but
> add that "reduced" ETA with the corresponding OMEGA equal to
> 0.01 or
> other small number. If the control stream code is correct, the
> objective function should decrease or retain the same value.
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
>
> web: www.quantpharm.com http://www.quantpharm.com
> http://www.quantpharm.com
>
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
> <tel:%28301%29%20767%205566>
>
> On 8/10/2013 10:23 PM, Xinting Wang wrote:
> > Dear all,
> >
> > Does anyone witnessed such a phenomenon in NONMEM as when you
> reduced an
> > ETA, the OFV value, rather than increase, actually decreased?
> It's quite
> > against intuition, as individual estimation should be better
> than
> > population estimation in that particular parameter. Both models,
> whether
> > having this ETA, converged very well.
> >
> > Best
> >
> > --
> > Xinting
>
> --
> Xinting
>
> --
>
> Xinting