Re: [External] Re: M3 method - WRES, and CWRES
I have a follow-up question on CWRES with M4 method.
I was able to run my model with M3 method, I got NPDE and CWRES (with MDVRES=1)
calculated just fine. Then I changed to M4 method by adding YLO to the non-BLQ
data, the model stilled converged but in the output table, some of the subjects
had CWRES = 0 and NPDE was a constant (around 3). The problem with CWRES and
NPDE is not specific to subjects with BLQ observations, but rather, it followed
a pattern like this: CWRES and NPDE was calculated for subjects number 1, 3, 5,
7, etc. and not calculated for subjects 2, 4, 6, 8, etc.
I suspected that YLO option was the cause, so I ran the model with M2 method.
Indeed, all CWRES was 0.
This might not be a new problem, but I searched through the NMusers archives
and most discussions focused on M3 only. Is there any reason why there is less
focus on M2 and M4?
Thank you
Quoted reply history
________________________________
From: [email protected] <[email protected]> on behalf of
Matthew Fidler <[email protected]>
Sent: Saturday, September 5, 2020 10:17 AM
To: Bauer, Robert <[email protected]>
Cc: [email protected] <[email protected]>
Subject: [External] Re: [NMusers] M3 method - WRES, and CWRES
Thank you Bob,
The NPDE 2.0 manual discusses the methods that NPDE uses to handle BLQ,
including replacing values with pred, ipred, or lloq, or simulating from a
uniform random value while calculating the NPDE (cdf method). The NONMEM
manual doesn't mention the method used. My guess is the cdf method.
I realize that no one has answered Mu'taz's question.
As far as if the CWRES is appropriate for BLQ data, the CWRES method uses the
FOCEi approximation to calculate residuals. However with M3/M4 and other
methods the likelihood for these points is not the FOCEi objective function but
the M3/M4 likelihood so anything you do here with CWRES doesn't follow or add
to the likelihood observed during minimization. Therefore in my opinion, there
will be bias of some sort here.
Best Regards,
Matt.
On Thu, Sep 3, 2020 at 3:06 PM Bauer, Robert
<[email protected]<mailto:[email protected]>> wrote:
Matt:
The NPDE and NPD systems in NONMEM are described in the nm744.pdf manual (
https://nonmem.iconplc.com/nonmem744 ), pages 70-75, and follow along the work
of Comet, Brendel, Ngyuen, Mentre, etc. The NPDE R package is not used within
NONMEM.
Robert J. Bauer, Ph.D.
Senior Director
Pharmacometrics R&D
ICON Early Phase
820 W. Diamond Avenue
Suite 100
Gaithersburg, MD 20878
Office: (215) 616-6428
Mobile: (925) 286-0769
[email protected]<mailto:[email protected]>
http://www.iconplc.com
From: [email protected]<mailto:[email protected]>
<[email protected]<mailto:[email protected]>> On Behalf
Of Matthew Fidler
Sent: Thursday, September 3, 2020 6:08 AM
To: Jeroen Elassaiss-Schaap (PD-value B.V.)
<[email protected]<mailto:[email protected]>>
Cc: Bill Denney
<[email protected]<mailto:[email protected]>>; Mu'taz
Jaber <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>
Subject: Re: [NMusers] M3 method - WRES, and CWRES
Hi everyone,
As an aside, nlmixr's upcoming release (that supports censoring) simulates a
value using a truncated normal based on the ipred, variance at that point and
the censoring column to produce an observation. This observation is used to
calculate RES, WRES, CWRES. It is flagged so you can see which values use this
approach. In theory, since this is simulated from the IPRED/truncated the
CWRES would be likely follow the distribution closer.
I'm unsure if the new NONMEM uses this approach.
Another question from my end is the NPDE: There are many methods to handle BLQ
values with NPDE R package, does anyone know which NONMEM uses? Or do you need
to use the NPDE package to get these values from NONMEM?
Matt.
On Wed, Sep 2, 2020 at 2:09 AM Jeroen Elassaiss-Schaap (PD-value B.V.)
<[email protected]<mailto:[email protected]>> wrote:
Hi Mutaz, Bill,
It might be useful to use NPDEs, as discussed in
https://www.cognigen.com/nmusers/2019-February/7376.html; the whole thread is
worthwhile reading. NPDEs can be calculated also for BQL values.
Bill -thanks for pointing to excellent post of Matt! I would take as most
important point that CWRES for non-BQL values, calculated with a model with
influential BQL, are biased because the influence of the BQL values is not
accounted for. (if a certain prediction for a measurable concentration is
changed by 10% because of the M3 method, that will turn up as a similar bias in
CWRES). The NPDEs as referenced to in the above discussion (Nguyen2012 JPKPD
0.1007/s10928-012-9264-2) do not suffer from that drawback as one can see the
complete profile (cf Fig 8 of Nguyen2012).
Hope this helps,
Jeroen
http://pd-value.com
[email protected]<mailto:[email protected]>
@PD_value
+31 6 23118438
-- More value out of your data!
On 2/9/20 2:32 am, Bill Denney wrote:
Hi Mutaz,
Matt Hutmacher described it well here:
https://www.cognigen.com/nmusers/2010-April/2448.html
A very brief summary of his excellent post is that subjects with a combination
of censored (BLQ) an uncensored (above the LLOQ and below the ULOQ) will be
biased in their reporting of CWRES because you cannot calculate CWRES for BLQ
values. (I say this before looking up what MDVRES does.)
My guess that Bob or someone else can confirm is that the bias is anticipated
to be relatively small compared to the value of being able to compare CWRES
values the other observations for a subject. It does not definitively mean
that the results are unbiased (see Matt’s Tmax example), but generally, the
CWRES values previously omitted are more useful than excluding them from
calculation.
Thanks,
Bill
From: [email protected]<mailto:[email protected]>
<[email protected]<mailto:[email protected]>> On Behalf
Of Mu'taz Jaber
Sent: Tuesday, September 1, 2020 7:25 PM
To: [email protected]<mailto:[email protected]>
Subject: [NMusers] M3 method - WRES, and CWRES
All,
Back in April 2010, Sebastian Bihorel and Martin Bergstrand initiated a
discussion regarding using the M3 and M4 methods for handling BQL data and how
it seemed to be a bug that NONMEM wouldn't compute WRES for the entire set of
subject data records whenever a BQL was included
( https://www.cognigen.com/nmusers/2010-April/2445.html). Tom Ludden responded
with the following post ( https://www.cognigen.com/nmusers/2010-April/2447.html):
This issue was discussed with Stuart Beal. He believed that weighted
residuals would be incorrect for an individual that had both continuous
dependent variables and a likelihood in the calculation of their
contribution to the objective function value, as is the case with his M3
or M4 BQL methods The code for both RES and WRES are intentionally
bypassed in these cases.
Since then, we now have easy functionality with the F_FLAG=1 condition of the
M3/M4 code in $ERROR to tack on MDVRES=1 that allows the calculation of WRES
and CWRES to be available in output tables.
My questions are: Is Stuart Beal's original concern still valid? Do these
NONMEM updates give us appropriate WRES and CWRES for plotting purposes for
individuals whose records contain BQL data?
Thank you,
Mutaz Jaber
PhD student
University of Minnesota
-------------------------------------------------------
Mutaz M. Jaber, PharmD.
PhD student, Pharmacometrics
Experimental and Clinical Pharmacology
University of Minnesota
717 Delaware St SE; Room 468
Minneapolis, MN 55414
Email: [email protected]<mailto:[email protected]>
Phone: +1 651-706-5202
~ Stay curious