Re: Reducing ETAs actually decreased OFV
I am not sure that you need likelihood profiling or any other sophisticated procedures to study this particular problem. You can look at relative standard errors of the parameter estimates: if one of the ETAs is poorly estimated, this is the candidate for removal. For two-compartment models, it is rarely possible to estimate ETAs on peripheral compartment, and at least one of those can be removed (usually).
If the goal is to describe the data, you look for the simplest model that allow you to fit the data. You may start with the model with all random effects, but then try to reduce the number of random effect (unless you use new IMP/SAEM/BAYES type procedures) to arrive at the simpler model. You may use OF as a guide: if OF drop is small when you remove the ETA, this ETA does not contribute to the fit (and the model can equally well fit the data without this particular ETA). Alternative procedure is to compare full (with ETAs) and reduced (with one ETA fixed to zero) model using various diagnostic plots procedure (VPC in particular), or plots of one model versus the other model: PRED vs PRED and IPRED vs IPRED (where PRED and IPRED belog to two models that you are comparing). If these plots looks like identity lines (both in normal and log axes), you can safely use simpler model, especially if VPC results are similar or identical.
As to the specific procedure that allowed you to fix the strange OF behavior, even the simple problems (like two-compartment model that was used) are highly nonlinear, and gradient methods cannot guarantee the global minimum. The solution (local minimum) may depend on initial conditions. By starting from the solution of the reduced problem, you put the model in the vicinity of the correct local minimum, while when you started from the larger model, it converged to the different minimum. This is not a universal procedure, but it helps time to time if the model has difficulties finding the solution.
Regards,
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 8/26/2013 9:58 AM, Denney, William S. wrote:
> Hi Xinting,
>
> This is a rather broad (and often highly-opinionated) topic. At the
> highest level, you can only fit parameters in a model where you have
> enough data to estimate the parameter. A simple example is that if you
> have data that you want to fit an Emax model to with measurements only
> up to the EC10, you don’t have enough data to estimate Emax and ED50; it
> will look fully linear.
>
> Due to data variability and the fact that Q and V3 are less correlated
> with the measurements than other parameters (Ka, V2, and CL have a
> stronger effect on the measurements than Q and V3), the estimates will
> be more difficult. A good example of how to evaluate this would be what
> Peter Bonate just suggested: do likelihood profiling on each of the
> parameters (especially the ETAs) to estimate the certainty (peakedness)
> or uncertainty (flatness) in the parameter estimates.
>
> Thanks,
>
> Bill
>
> *From:*Xinting Wang [mailto:[email protected]]
> *Sent:* Monday, August 26, 2013 9:27 AM
> *To:* Leonid Gibiansky
> *Cc:* [email protected]; Denney, William S.
> *Subject:* Re: [NMusers] Reducing ETAs actually decreased OFV
>
> Dear Bill,
>
> Appreciate your reply a lot. The issue is from KA. Adding KA or not did
> have this problem. However, regarding your statement "it is rare to have
> enough data to fit true IIV", can you explain more about this. My data
> set is from Phase I studies, and I thought this should be enough for
> this simulation.
>
> Dear Leonid,
>
> Thanks very much for your detailed suggestion. I followed the steps you
> listed above, and did find that the OFV decreased in step 2, just as you
> predicted. Then using the estimation to replace the initial values for
> all of the THETA, OMEGA and SIGMA, the OFV stabilized. However, I am
> curious about the explanation for this. And, is this a universal method
> for estimation of initial values? Thank you.
>
> The change of OFV was around 100 (the OFV was ~114300). I am pasting the
> OMEGA matrix below for your information.
>
> ETA1 ETA2 ETA3 ETA4 ETA5
>
> ETA1
> + 4.08E-02
>
> ETA2
> + 0.00E+00 1.57E-01
>
> ETA3
> + 0.00E+00 0.00E+00 1.30E-01
>
> ETA4
> + 0.00E+00 0.00E+00 0.00E+00 4.07E-01
>
> ETA5
> + 0.00E+00 0.00E+00 0.00E+00 0.00E+00 2.19E-02
>
> ETA5 (0.0219) is the one caused the problem.
>
> Best Regards
>
> On 26 August 2013 07:06, Leonid Gibiansky <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Xinting,
> You should be able to do it. Let's check it again this way
> 1. You run the model with all ETAs included, but one ETA (the one that
> was excluded in the reduced model) is fixed to zero. You should be able
> to reproduce your "reduced ETA" result (OF)
> 2. You take the same control stream, and set all initial values to the
> final parameter estimates of model (1) above, except you use the small
> value (may be not 0.01 but 0.000001) as the initial value of the ETA
> that was fixed to zero in model (1).
>
> Model (2) is the not-reduced model, and it's OF should be less or equal
> to the OF of model (1). If this is not the case, increase the number of
> significant digits in the initial estimates of model (2) - take those
> from the final estimates of model 1.
>
> Without data, it is very difficult to offer more specific advice.
>
> Also, what is the magnitude of the OF change? What is the estimate of
> the OMEGA for the ETA in question?
>
> Regards,
>
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web: www.quantpharm.com http://www.quantpharm.com
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
>
> On 8/25/2013 8:42 AM, Xinting Wang wrote:
>
> Dear Leonid,
>
> I tried with your method and found the same result. The initial
> estimation of the added ETA was set at 0.01, and the result showed an
> increase of OFV. Please see below the $PK part of the control file for
> more information. Many thanks.
>
> Dear Bill,
>
> Could you please explain that in a little bit more detail? I am pasting
> the $PK part of the control file in case you could find the useful
> information. Thanks a lot.
>
> $PK
>
> FA1=0
> FA2=0
> FA3=0
> FA4=0
>
> IF(DOSE.EQ.250) THEN
> FA1=1
> ENDIF
>
> IF(DOSE.EQ.500) THEN
> FA2=1
> ENDIF
>
> IF(DOSE.EQ.850) THEN
> FA3=1
> ENDIF
>
> IF(DOSE.EQ.1000) THEN
> FA4=1
> ENDIF
>
> F1=FA1+FA2*THETA(6)+FA3*THETA(7)+FA4*THETA(8)
>
> TVCL=THETA(1)
> TVV2=THETA(2)
> TVKA=THETA(3)
> TVQ=THETA(4)
> TVV3=THETA(5)
>
> CL=TVCL*EXP(ETA(1))
> V2=TVV2*EXP(ETA(2))
> KA=TVKA*EXP(ETA(5))
> Q=TVQ*EXP(ETA(3))
> V3=TVV3*EXP(ETA(4))
>
> S2=V2/1000
> S3=V3/1000
>
> $ERROR
>
> IPRE=F
>
> IRES=DV-IPRE
>
> W=F
>
> IF(W.EQ.0) W = 1
>
> IWRE = IRES/W
>
> Y=F*(1+EPS(1))+EPS(2)
>
> Best Regards
>
> On 12 August 2013 20:50, Denney, William S.
> <[email protected] <mailto:[email protected]>
>
> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
>
> Hi Xinting,
>
> In a few rare cases, I've seen this happen if the model is
> approaching nonconvergence. In those cases, typically the RSE on
> one or more parameters will increase and the ratio of max to min
> eigenvalues will increase substantially. Are you seeing either of
> these?
>
> Thanks,
>
> Bill
>
> On Aug 11, 2013, at 21:56, "Leonid Gibiansky"
>
> <[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
>
> Xinting,
> Try to start from the initial conditions of your "reduced"
> model but
> add that "reduced" ETA with the corresponding OMEGA equal to
> 0.01 or
> other small number. If the control stream code is correct, the
> objective function should decrease or retain the same value.
> Leonid
>
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
>
> web: www.quantpharm.com http://www.quantpharm.com
> http://www.quantpharm.com
>
> e-mail: LGibiansky at quantpharm.com http://quantpharm.com
> http://quantpharm.com
> tel: (301) 767 5566 <tel:%28301%29%20767%205566>
> <tel:%28301%29%20767%205566>
>
> On 8/10/2013 10:23 PM, Xinting Wang wrote:
> > Dear all,
> >
> > Does anyone witnessed such a phenomenon in NONMEM as when you
> reduced an
> > ETA, the OFV value, rather than increase, actually decreased?
> It's quite
> > against intuition, as individual estimation should be better
> than
> > population estimation in that particular parameter. Both models,
> whether
> > having this ETA, converged very well.
> >
> > Best
> >
> > --
> > Xinting
>
> --
> Xinting
>
> --
>
> Xinting