Re: Calculating shrinkage when some etas are zero
Hello Jakob, et al.
I would agree that individuals who do not contribute data to the estimation of a particular element of OMEGA should be excluded from the ETA-shrinkage calculation or ETA-based diagnostics. I think that using individual ETA=0 as the filtering criterion may be a reasonable thing to do when OMEGA is DIAGONAL (e.g. all off-diagonal elements are zero), but this practice could be misleading when covariance in the inter-individual random effects exists (e.g. OMEGA BLOCK(n)).
For example, consider a population PK model simultaneously incorporating parent and metabolite data. Also imagine that the OMEGA matrix is constructed to allow covariance between ETA[parent CL] and ETA[metabolite CL]. If the correlation between these ETAs is non-zero, it is possible that individuals who are entirely missing metabolite data will still have a non-zero ETA[metabolite CL] estimate. This is because the expected value for that ETA should be driven by the covariance structure in OMEGA. Although this ETA estimate is non-zero, it is shrunken toward the population expected value, and may contribute to a biased shrinkage calculation and/or diagnostics.
To avoid both this situation and the issue that Douglas raised, it is preferable to filter ETAs based on design factors rather than automatically based on individual ETA=0.
Having said all this, I'm not sure how important this particular source of bias in the ETA-shrinkage calculation is anyway. There are other potential biases in this calculation, including: 1. Bias in the population estimates of OMEGA variance elements- It's not uncommon for these terms to be over-estimated, which may lead to an artificial apparent shrinkage (the calculation for ETA shrinkage uses estimated variance in the denominator). 2. Bias in the observed sample SD of individual ETAs due to insufficient sample size- Biased shrinkage estimates may result from biased sample SD (used in the numerator of the shrinkage calculation), particularly in small data sets.
I think the take-home message is that ETA-based diagnostics (and diagnostics of the diagnostics) can be useful, but should be considered in the context of the design and potential biases.
Best regards,
Marc
Marc R. Gastonguay, Ph.D. < [email protected] >
President & CEO, Metrum Research Group LLC < metrumrg.com >
Scientific Director, Metrum Institute < metruminstitute.org >
2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct: +1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014
Quoted reply history
On Aug 21, 2009, at 9:12 AM, Ribbing, Jakob wrote:
> Hi Douglas,
>
> This has been a concern for me as well, although I do not know if this ever happens(?). For the automatic (generic scripts) exclusion of etas that I use for eta-diagnostics, I tend to exclude a group (e.g. each dose or dose-study combination) if all subjects have eta=0 in that group. This would for example exclude IOV-eta3 from a study that only hade two occasions, or the placebo group(s) for etas on drug effect. I feel safe with that exclusion for my diagnostics. If I had to make the choice between excluding all etas that are exactly equal to zero or none at all, I would more trust diagnostics after exclusion.
>
> Jakob
>
> From: Eleveld, DJ [mailto:[email protected]]
> Sent: 21 August 2009 13:57
> To: Ribbing, Jakob; Pyry Välitalo; [email protected]
> Subject: RE: [NMusers] Calculating shrinkage when some etas are zero
>
> Hi Pyry and Jacob,
>
> If you exclude zero etas then what happens to infomative individuals who just happen to have the population typical values? This approch would exclude these individuals when trying to indicate how informative an estimation is about a parameter.
>
> I know this is unlikely, but it is possible.
>
> The etas just tell what value is estimated, its not the whole story about how infomative an estimation is. I dont think you can do this without considering how 'certian' you are of each of those eta values.
>
> Douglas Eleveld
>
> Van: [email protected] namens Ribbing, Jakob
> Verzonden: vr 21-8-2009 12:26
> Aan: Pyry Välitalo; [email protected]
> Onderwerp: RE: [NMusers] Calculating shrinkage when some etas are zero
>
> Hi Pyry,
>
> Yes, when calculating shrinkage or looking at eta-diagnostic plots it is often better to exclude etas from subjects that has no information on that parameter at all. For a PK model we would not include subjects that were only administered placebo (if PK is exogenous compound). In the same manner placebo subjects are not informative on the drug-effects parameters of a (PK-)PD model. These subjects have informative etas for the placebo-part of the PD model, but not on the drug-effects (etas on Emax, ED50, etc.). For any eta- diagnostics you can removed these etas based on design (placebo subject, IV dosing, et c) or the empirical-Bayes estimate of eta being zero.
>
> Cheers
>
> Jakob
>
> From: [email protected] [ mailto: [email protected] ] On Behalf Of Pyry Välitalo
>
> Sent: 21 August 2009 10:45
> To: [email protected]
> Subject: [NMusers] Calculating shrinkage when some etas are zero
>
> Hi all,
>
> I saw this snippet of information on PsN-general mailing list.
>
> Kajsa Harling wrote in PsN-general:
>
> "I talked to the experts here about shrinkage. Apparently, sometimes an
>
> individual's eta may be exactly 0 (no effect, placebo, you probably
>
> understand this better than I do). These zeros should not be included in
>
> the shrinkage calculation, but now they are (erroneously) in PsN."
>
> This led me to wonder about the calculation of shrinkage. I decided to post here on nmusers, because my question mainly relates to NONMEM. I could not find previous discussions about this topic exactly.
>
> As I understand, if a parameter with BSV is not used by some individuals, the etas for these individuals will be set to zero. An example would be a dataset with IV and oral dosing data. If oral absorption rate constant KA with BSV is estimated for this data, then all eta(KA) values for IV dosing group will be zero.
>
> The shrinkage of etas is calculated as
> 1-sd(etas)/omega
>
> If the etas that equal exactly zero would have to be removed from this equation then it would mean that NONMEM estimates the omega based on only those individuals who need it for the parameter in question, e.g. the omega(KA) would be estimated only based on the oral dosing group. Is this a correct interpretation for the rationale to leave out zero etas?
>
> I guess the inclusion of zero etas into shrinkage calculations significantly increases the estimate of shrinkage because the zero etas always reduce the sd(etas). As a practical example, suppose a dataset of 20 patients with oral and 20 patients with IV administration. Suppose NONMEM estimates an omega of 0.4 for BSV of KA. Suppose the sd(etas) for oral group is 0.3 and thus sd(etas) for all patients is 0.3/sqrt(2) since the etas in IV group for KA are zero. Thus, as far as I know, PsN would currently calculate a shrinkage of 1-(0.3/sqrt(2))/0.4=0.47. Would it be more appropriate to manually calculate a shrinkage of 1-0.3/0.4=0.25 instead?
>
> All comments much appreciated.
>
> Kind regards,
> Pyry
>
> Kajsa Harling wrote:
> Dear Ethan,
>
> I have also been away for a while, thank you for your patience.
>
> I talked to the experts here about shrinkage. Apparently, sometimes an
> individual's eta may be exactly 0 (no effect, placebo, you probably
>
> understand this better than I do). These zeros should not be included in
>
> the shrinkage calculation, but now they are (erroneously) in PsN.
>
> Does this explain the discrepancy?
>
> Then, the heading shrinkage_wres is incorrect, it should say
> shrinkage_iwres (or eps) they say.
>
> Comments are fine as long as they do not have commas in them. But this
> is fixed in the latest release.
>
> Best regards,
> Kajsa