Re: Calculating shrinkage when some etas are zero

On 12/07/2010 11:16 AM, [email protected] wrote: > Hi to all, > > Do you know if there is a quick method to exclude subjects with ETA=0 > from the calculation of ETA shrinkage using NONMEM 7? > > I also tried to use the option –shrinkage of PsN, but I get the > following error: > > AN ERROR WAS FOUND IN THE CONTROL STATEMENTS. > > 187 $TABLE RECORD REQUESTS AN UNKNOWN ITEM. > > at /usr/lib/perl5/site_perl/5.8.8/PsN_3_2_4/nonmem.pm line 40 > > Kind Regards > > Marco > > ------------------------------------------------------------------------------ > Marco Campioni, PhD > Modelling & Simulation Senior Scientist > Exploratory Medicine > > Merck Serono S.A. - Geneva > > *"Gastonguay, Marc" <[email protected]>* > Sent by: [email protected] > > 21/08/2009 18:10 > > To > "Ribbing, Jakob" <[email protected]> > cc > "Eleveld, DJ" <[email protected]>, Pyry Välitalo > <[email protected]>, <[email protected]> > Subject > Re: [NMusers] Calculating shrinkage when some etas are zero > > Hello Jakob, et al. > > I would agree that individuals who do not contribute data to the > estimation of a particular element of OMEGA should be excluded from the > ETA-shrinkage calculation or ETA-based diagnostics. I think that using > individual ETA=0 as the filtering criterion may be a reasonable thing to > do when OMEGA is DIAGONAL (e.g. all off-diagonal elements are zero), but > this practice could be misleading when covariance in the > inter-individual random effects exists (e.g. OMEGA BLOCK(n)). > > For example, consider a population PK model simultaneously incorporating > parent and metabolite data. Also imagine that the OMEGA matrix is > constructed to allow covariance between ETA[parent CL] and > ETA[metabolite CL]. If the correlation between these ETAs is non-zero, > it is possible that individuals who are entirely missing metabolite data > will still have a non-zero ETA[metabolite CL] estimate. This is because > the expected value for that ETA should be driven by the covariance > structure in OMEGA. Although this ETA estimate is non-zero, it is > shrunken toward the population expected value, and may contribute to a > biased shrinkage calculation and/or diagnostics. > > To avoid both this situation and the issue that Douglas raised, it is > preferable to filter ETAs based on design factors rather than > automatically based on individual ETA=0. > > Having said all this, I'm not sure how important this particular source > of bias in the ETA-shrinkage calculation is anyway. There are other > potential biases in this calculation, including: > 1. Bias in the population estimates of OMEGA variance elements- It's not > uncommon for these terms to be over-estimated, which may lead to an > artificial apparent shrinkage (the calculation for ETA shrinkage uses > estimated variance in the denominator). > 2. Bias in the observed sample SD of individual ETAs due to insufficient > sample size- Biased shrinkage estimates may result from biased sample SD > (used in the numerator of the shrinkage calculation), particularly in > small data sets. > > I think the take-home message is that ETA-based diagnostics (and > diagnostics of the diagnostics) can be useful, but should be considered > in the context of the design and potential biases. > > Best regards, > Marc > > Marc R. Gastonguay, Ph.D. < [email protected]_ > <mailto:[email protected]> > > President & CEO, Metrum Research Group LLC < metrumrg.com > > Scientific Director, Metrum Institute < metruminstitute.org > > 2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct: +1.860.670.0744 > Main: +1.860.735.7043 Fax: +1.860.760.6014 > > On Aug 21, 2009, at 9:12 AM, Ribbing, Jakob wrote: > > Hi Douglas, > > This has been a concern for me as well, although I do not know if this > ever happens(?). For the automatic (generic scripts) exclusion of etas > that I use for eta-diagnostics, I tend to exclude a group (e.g. each > dose or dose-study combination) if all subjects have eta=0 in that > group. This would for example exclude IOV-eta3 from a study that only > hade two occasions, or the placebo group(s) for etas on drug effect. I > feel safe with that exclusion for my diagnostics. If I had to make the > choice between excluding all etas that are exactly equal to zero or none > at all, I would more trust diagnostics after exclusion. > > Jakob > > ------------------------------------------------------------------------ > > *From:* Eleveld, DJ [_mailto:[email protected]_] * > Sent:* 21 August 2009 13:57* > To:* Ribbing, Jakob; Pyry Välitalo; [email protected]_ > <mailto:[email protected]>* > Subject:* RE: [NMusers] Calculating shrinkage when some etas are zero > > Hi Pyry and Jacob, > > If you exclude zero etas then what happens to infomative individuals who > just happen to have the population typical values? > This approch would exclude these individuals when trying to indicate how > informative an estimation is about a parameter. > I know this is unlikely, but it is possible. > > The etas just tell what value is estimated, its not the whole story > about how infomative an estimation is. I dont think you can do > this without considering how 'certian' you are of each of those eta values. > > Douglas Eleveld > > ------------------------------------------------------------------------ > > *Van:* [email protected]_ > <mailto:[email protected]> namens Ribbing, Jakob* > Verzonden:* vr 21-8-2009 12:26* > Aan:* Pyry Välitalo; [email protected]_ > <mailto:[email protected]>* > Onderwerp:* RE: [NMusers] Calculating shrinkage when some etas are zero > Hi Pyry, > > Yes, when calculating shrinkage or looking at eta-diagnostic plots it is > often better to exclude etas from subjects that has no information on > that parameter at all. For a PK model we would not include subjects that > were only administered placebo (if PK is exogenous compound). In the > same manner placebo subjects are not informative on the drug-effects > parameters of a (PK-)PD model. These subjects have informative etas for > the placebo-part of the PD model, but not on the drug-effects (etas on > Emax, ED50, etc.). For any eta-diagnostics you can removed these etas > based on design (placebo subject, IV dosing, et c) or the > empirical-Bayes estimate of eta being zero. > > Cheers > > Jakob > > ------------------------------------------------------------------------ > > *From:* [email protected]_ > <mailto:[email protected]> > [_mailto:[email protected]_] *On Behalf Of *Pyry Välitalo* > Sent:* 21 August 2009 10:45* > To:* [email protected]_ <mailto:[email protected]>* > Subject:* [NMusers] Calculating shrinkage when some etas are zero > > Hi all, > > I saw this snippet of information on PsN-general mailing list. > > Kajsa Harling wrote in PsN-general: > "I talked to the experts here about shrinkage. Apparently, sometimes an > individual's eta may be exactly 0 (no effect, placebo, you probably > understand this better than I do). These zeros should not be included in > the shrinkage calculation, but now they are (erroneously) in PsN." > > This led me to wonder about the calculation of shrinkage. I decided to > post here on nmusers, because my question mainly relates to NONMEM. I > could not find previous discussions about this topic exactly. > > As I understand, if a parameter with BSV is not used by some > individuals, the etas for these individuals will be set to zero. An > example would be a dataset with IV and oral dosing data. If oral > absorption rate constant KA with BSV is estimated for this data, then > all eta(KA) values for IV dosing group will be zero. > > The shrinkage of etas is calculated as > 1-sd(etas)/omega > If the etas that equal exactly zero would have to be removed from this > equation then it would mean that NONMEM estimates the omega based on > only those individuals who need it for the parameter in question, e.g. > the omega(KA) would be estimated only based on the oral dosing group. Is > this a correct interpretation for the rationale to leave out zero etas? > > I guess the inclusion of zero etas into shrinkage calculations > significantly increases the estimate of shrinkage because the zero etas > always reduce the sd(etas). As a practical example, suppose a dataset of > 20 patients with oral and 20 patients with IV administration. Suppose > NONMEM estimates an omega of 0.4 for BSV of KA. Suppose the sd(etas) for > oral group is 0.3 and thus sd(etas) for all patients is 0.3/sqrt(2) > since the etas in IV group for KA are zero. > Thus, as far as I know, PsN would currently calculate a shrinkage of > 1-(0.3/sqrt(2))/0.4=0.47. > Would it be more appropriate to manually calculate a shrinkage of > 1-0.3/0.4=0.25 instead? > > All comments much appreciated. > > Kind regards, > Pyry > > Kajsa Harling wrote: > Dear Ethan, > > I have also been away for a while, thank you for your patience. > > I talked to the experts here about shrinkage. Apparently, sometimes an > individual's eta may be exactly 0 (no effect, placebo, you probably > understand this better than I do). These zeros should not be included in > the shrinkage calculation, but now they are (erroneously) in PsN. > > Does this explain the discrepancy? > > Then, the heading shrinkage_wres is incorrect, it should say > shrinkage_iwres (or eps) they say. > > Comments are fine as long as they do not have commas in them. But this > is fixed in the latest release. > > Best regards, > Kajsa > > This message and any attachment are confidential and may be privileged > or otherwise protected from disclosure. If you are not the intended > recipient, you must not copy this message or attachment or disclose the > contents to any other person. If you have received this transmission in > error, please notify the sender immediately and delete the message and > any attachment from your system. Merck KGaA, Darmstadt, Germany and any > of its subsidiaries do not accept liability for any omissions or errors > in this message which may arise as a result of E-Mail-transmission or > for damages resulting from any unauthorized changes of the content of > this message and any attachment thereto. Merck KGaA, Darmstadt, Germany > and any of its subsidiaries do not guarantee that this message is free > of viruses and does not accept liability for any damages caused by any > virus transmitted therewith. > > Click _ http://disclaimer.merck.de_ to access the German, French, Spanish > and Portuguese versions of this disclaimer. -- ---------------------------------------- Kajsa Harling, PhD System Developer Department of Pharmaceutical Biosciences Uppsala University [email protected] +46-(0)18-471 4291