Dear NMUSERS,
A few years back, there was a discussion on the P-value for ETABAR. However, I
am not sure how to appropriately handle very small P-value(s) for ETABAR
situation during the development of a model.
I need some clarifications to a few questions:
1: Should we just ignore this small P-Value warning?
2: Can we change IIV model to avoid small P-value for ETABAR? Or any other
suggestions?
3: Does NONMEM make any assumptions on ETA distribution?
This P-value for ETABAR really bugs me a lot. I look forward to seeing some
input.
Thank you and I appreicate your time and help.
Jian
Very small P-Value for ETABAR
Hi Jian,
I would look for a covariate effect on that parameter.
Thanks,
Bill
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Jian Xu
Sent: Thursday, November 13, 2008 10:16 AM
To: [email protected]
Subject: [NMusers] Very small P-Value for ETABAR
Dear NMUSERS,
A few years back, there was a discussion on the P-value for ETABAR.
However, I am not sure how to appropriately handle very small P-value(s)
for ETABAR situation during the development of a model.
I need some clarifications to a few questions:
1: Should we just ignore this small P-Value warning?
2: Can we change IIV model to avoid small P-value for ETABAR? Or any
other suggestions?
3: Does NONMEM make any assumptions on ETA distribution?
This P-value for ETABAR really bugs me a lot. I look forward to seeing
some input.
Thank you and I appreicate your time and help.
Jian
Notice: This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates (which may be known
outside the United States as Merck Frosst, Merck Sharp & Dohme or
MSD and in Japan, as Banyu - direct contact information for affiliates is
available at http://www.merck.com/contact/contacts.html) that may be
confidential, proprietary copyrighted and/or legally privileged. It is
intended solely for the use of the individual or entity named on this
message. If you are not the intended recipient, and have received this
message in error, please notify us immediately by reply e-mail and
then delete it from your system.
You might want to look at the shrinkage for the eta in question.
Regards,
Pankaj
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Denney, William S.
Sent: Thursday, November 13, 2008 10:41 AM
To: Jian Xu; [email protected]
Subject: RE: [NMusers] Very small P-Value for ETABAR
Hi Jian,
I would look for a covariate effect on that parameter.
Thanks,
Bill
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Jian Xu
Sent: Thursday, November 13, 2008 10:16 AM
To: [email protected]
Subject: [NMusers] Very small P-Value for ETABAR
Dear NMUSERS,
A few years back, there was a discussion on the P-value for ETABAR.
However, I am not sure how to appropriately handle very small P-value(s)
for ETABAR situation during the development of a model.
I need some clarifications to a few questions:
1: Should we just ignore this small P-Value warning?
2: Can we change IIV model to avoid small P-value for ETABAR? Or any
other suggestions?
3: Does NONMEM make any assumptions on ETA distribution?
This P-value for ETABAR really bugs me a lot. I look forward to seeing
some input.
Thank you and I appreicate your time and help.
Jian
Notice: This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates (which may be known
outside the United States as Merck Frosst, Merck Sharp & Dohme or
MSD and in Japan, as Banyu - direct contact information for affiliates
is
available at http://www.merck.com/contact/contacts.html) that may be
confidential, proprietary copyrighted and/or legally privileged. It is
intended solely for the use of the individual or entity named on this
message. If you are not the intended recipient, and have received this
message in error, please notify us immediately by reply e-mail and
then delete it from your system.
Hi Jian,
As Bill says, including a covariate may fix your problem. However, two
other underlying problems may also be causing this:
1. Asymmetric shrinkage of the eta. Two examples of this that I
have seen is if you have an eta on epsilon (different residual-error
magnitude in different subjects) or if the doses yield a clear effect in
some subjects but not in others (eta on EC50/EC50 may become more shrunk
on the right tail, since any drug effect in the less sensitive subjects
is difficult to separate from the background noise or circadian
variation). An important covariate may reduce the degree of shrinkage
and the asymmetry in the shrinkage. Other than that, shrinkage is not an
issue unless you use the empirical Bayes estimates for diagnostics, i.e.
use the individual parameters in graphs, calculations, PK predictions as
input to the PD model (IPK approach), etc.
2. Incorrect distributional assumptions: The parametric model
assumes e.g. a log-normal distribution of the parameter, around its
typical value. If this is not correct eta bar may become biased. You may
try other transformations in nonmem, e.g. proportional or other,
so-called semi-parametric distributions.
For references on Semi-parameteric distributions, search abstracts from
Petterson, Hanze, Savic and Karlsson. For reference on shrinkage, see
the publication below.
Cheers
Jakob
Clin Pharmacol Ther. 2007 Jul;82(1):17-20. Links
Diagnosing model diagnostics.Karlsson MO, Savic RM.
Department of Pharmaceutical Biosciences, Uppsala University, Uppsala,
Sweden.
Conclusions from clinical trial results that are derived from
model-based analyses rely on the model adequately describing the
underlying system. The traditionally used diagnostics intended to
provide information about model adequacy have seldom discussed
shortcomings. Without an understanding of the properties of these
diagnostics, development and use of new diagnostics, and additional
information pertaining to the diagnostics, there is risk that adequate
models will be rejected and inadequate models accepted. Thus, a
diagnosis of available diagnostics is desirable.
Quoted reply history
________________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Denney, William S.
Sent: 13 November 2008 15:41
To: Jian Xu; [email protected]
Subject: RE: [NMusers] Very small P-Value for ETABAR
Hi Jian,
I would look for a covariate effect on that parameter.
Thanks,
Bill
________________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Jian Xu
Sent: Thursday, November 13, 2008 10:16 AM
To: [email protected]
Subject: [NMusers] Very small P-Value for ETABAR
Dear NMUSERS,
A few years back, there was a discussion on the P-value for ETABAR.
However, I am not sure how to appropriately handle very small P-value(s)
for ETABAR situation during the development of a model.
I need some clarifications to a few questions:
1: Should we just ignore this small P-Value warning?
2: Can we change IIV model to avoid small P-value for ETABAR? Or any
other suggestions?
3: Does NONMEM make any assumptions on ETA distribution?
This P-value for ETABAR really bugs me a lot. I look forward to seeing
some input.
Thank you and I appreicate your time and help.
Jian
Notice: This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates (which may be known
outside the United States as Merck Frosst, Merck Sharp & Dohme or
MSD and in Japan, as Banyu - direct contact information for affiliates
is
available at http://www.merck.com/contact/contacts.html) that may be
confidential, proprietary copyrighted and/or legally privileged. It is
intended solely for the use of the individual or entity named on this
message. If you are not the intended recipient, and have received this
message in error, please notify us immediately by reply e-mail and
then delete it from your system.
Another possible reason could be that you have a huge number of subjects
in your database. Then a tiny deviation from zero will generate a highly
significant p-value (very small). You should look at the estimate itself
too (ETABAR).
Yaning Wang, Ph.D.
Team Leader, Pharmacometrics
Office of Clinical Pharmacology
Office of Translational Science
Center for Drug Evaluation and Research
U.S. Food and Drug Administration
Phone: 301-796-1624
Email: [EMAIL PROTECTED]
"The contents of this message are mine personally and do not necessarily
reflect any position of the Government or the Food and Drug
Administration."
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Jian Xu
Sent: Thursday, November 13, 2008 11:32 AM
To: Denney, William S.; NM Users
Subject: Re: [NMusers] Very small P-Value for ETABAR
Hi, Bill,
Thanks for your input. I also got some answers from Mahesh, Yaning, and
Pankaj. Thank you guys!
This small P-value for ETABAR could be due to possible two reasons:
1: As Bill and Yaning suggested, this ETA is non-nomal distribuated,
which could result from covariates (e.g. gender) or mixture of
population (e.g. race)
2: As Mahesh and Pankaj suggested, the shrinkage of this ETA needs to be
calculated, which can help identify whether the model is ill
conditioning or overparameterized .
Cheers,
Jian
________________________________
From: "Denney, William S." <[EMAIL PROTECTED]>
To: Jian Xu <[EMAIL PROTECTED]>; [email protected]
Sent: Thursday, November 13, 2008 10:41:15 AM
Subject: RE: [NMusers] Very small P-Value for ETABAR
Hi Jian,
I would look for a covariate effect on that parameter.
Thanks,
Bill
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Jian Xu
Sent: Thursday, November 13, 2008 10:16 AM
To: [email protected]
Subject: [NMusers] Very small P-Value for ETABAR
Dear NMUSERS,
A few years back, there was a discussion on the P-value for ETABAR.
However, I am not sure how to appropriately handle very small P-value(s)
for ETABAR situation during the development of a model.
I need some clarifications to a few questions:
1: Should we just ignore this small P-Value warning?
2: Can we change IIV model to avoid small P-value for ETABAR? Or any
other suggestions?
3: Does NONMEM make any assumptions on ETA distribution?
This P-value for ETABAR really bugs me a lot. I look forward to seeing
some input.
Thank you and I appreicate your time and help.
Jian
Notice: This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates (which may be known
outside the United States as Merck Frosst, Merck Sharp & Dohme or
MSD and in Japan, as Banyu - direct contact information for affiliates
is
available at http://www.merck.com/contact/contacts.html) that may be
confidential, proprietary copyrighted and/or legally privileged. It is
intended solely for the use of the individual or entity named on this
message. If you are not the intended recipient, and have received this
message in error, please notify us immediately by reply e-mail and
then delete it from your system.
Jian,
ETABAR p-value could be a useful quick check of the results, but it should not be used to replace the graphical evaluation of the model. Graphical evaluation should always include the histograms of the random effects, QQ plot of the random effects versus standard normal distribution, scatter plot matrix that visualizes correlation of the random effects, and plots of random effects versus all important (or all available) continuous and categorical covariates (as box-plots for categorical). These visual checks are much more powerful tools to detect a problem than ETABAR p-values. If they are OK than I would not worry about ETABAR. However, small ETABAR p-value often hints that the ETA distribution is not symmetric, or has outliers (non-symmetric long tails).
Thanks
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Jian Xu wrote:
> Dear NMUSERS,
>
> A few years back, there was a discussion on the P-value for ETABAR. However, I am not sure how to appropriately handle very small P-value(s) for ETABAR situation during the development of a model. I need some clarifications to a few questions:
>
> 1: Should we just ignore this small P-Value warning?
>
> 2: Can we change IIV model to avoid small P-value for ETABAR? Or any other suggestions?
>
> 3: Does NONMEM make any assumptions on ETA distribution?
>
> This P-value for ETABAR really bugs me a lot. I look forward to seeing some input. Thank you and I appreicate your time and help. Jian
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and
Rada that says ETABAR can be biased when there is a lot of shrinkage
even when the data is simulated and fitted with the correct model. Can
you confirm this and can you explain how it arises? In the worst case of
shrinkage then bias is impossible because all ETAs must be zero. So why
does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas having
> a normal distribution when estimating a parametric model in NONMEM. The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal, but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case: http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
Quoted reply history
> -----Original Message-----
> From: BAE, KYUN-SEOP
> Sent: 13 November 2008 22:05
> To: Ribbing, Jakob; XIA LI; nmusers
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal,
> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
> covariate relationships as others suggested,
> I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> ==========
> Kyun-Seop Bae MD PhD
> Email: kyun-seop.bae
>
> -----Original Message-----
> From: owner-nmusers
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; nmusers
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: owner-nmusers
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: nmusers
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
> significant test. You reject the null hypothesis.
>
> More...
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter equals
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
> of individual parameter will be changed to:
>
> VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ===================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Dear All,
Just some quick statistical points...
P value is usually associated with hypothesis test. As far as I know, NONMEM
assume normal distribution for ETA, ETA~N(0,omega), which means the null
hypothesis to test is H0: ETABAR=0. A small P value indicates a significant
test. You reject the null hypothesis.
More...
As we all know, ETA is used to capture the variation among individual
parameters and model's unexplained error. We usually use the function (or
model) parameter=typical value*exp (ETA), which leads to a lognormal
distribution assumption for all fixed effect parameters (i.e., CL, V, Ka,
Ke...).
By some statistical theory, the variation of individual parameter equals a
function of the typical value and the variance of ETA.
VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
If your typical value captures all overall patterns among patients clearance,
then ETA will have a nice symmetric normal distribution with small variance.
Otherwise, you leave too many patterns to ETA and will see some deviation or
shrinkage (whatever you call).
Why adding covariates is a good way to deal with this situation? You model
become CL=typical value*exp (covariate)*exp (ETA). The variation of individual
parameter will be changed to:
VAR (CL) = (typical value + covariate)*exp (omega/2)).
You have one more item to capture the overall patterns, less leave to ETA. So a
'good' covariate will reduce both the magnitude of omega and ETA's deviation
from normal.
Understanding this is also useful when you are modeling BOV studies. When you
see variation of PK parameters decrease with time (or occasions). Adding a
covariate that make physiological sense and also decrease with time may help
your modeling.
Best,
Xia
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
Dear All,
Realized etas (EBEs, MAPs) is estimated under the assumption of normal
distribution.
However, the resultant distribution of EBEs may not be normal or mean of
them may not be 0.
To pass t-test, one may use "CENTERING" option at $ESTIMATION.
But, this practice is discouraged by some (and I agree).
Normal assumption cannot coerce the distribution of EBE to be normal,
and furthermore non-normal (and/or not-zero-mean) distribution of EBE
can be nature's nature.
One simple example is mixture population with polymorphism.
If I could not get normal(?) EBEs even after careful examination of
covariate relationships as others suggested,
I would bear it and assume nonparametric distribution.
Regards,
Kyun-Seop
=====================
Kyun-Seop Bae MD PhD
Email: [EMAIL PROTECTED]
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Ribbing, Jakob
Sent: Thursday, November 13, 2008 13:19
To: XIA LI; [email protected]
Subject: RE: [NMusers] Very small P-Value for ETABAR
Hi Xia,
Just to clarify one thing (I agree with almost everything you said):
The p-value indeed is related to the test of ETABAR=0. However, this is
not a test of normality, only a test that may reject the mean of the
etas being zero (H0). Therefore, shrinkage per se does not lead to
rejection of HO, as long as both tails of the eta distribution are
shrunk to a similar degree.
I agree with the assumption of normality. This comes into play when you
simulate from the model and if you got the distribution of individual
parameters wrong, simulations may not reflect even the data used to fit
the model.
Best Regards
Jakob
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of XIA LI
Sent: 13 November 2008 20:31
To: [email protected]
Subject: Re: [NMusers] Very small P-Value for ETABAR
Dear All,
Just some quick statistical points...
P value is usually associated with hypothesis test. As far as I know,
NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
significant test. You reject the null hypothesis.
More...
As we all know, ETA is used to capture the variation among individual
parameters and model's unexplained error. We usually use the function
(or model) parameter=typical value*exp (ETA), which leads to a lognormal
distribution assumption for all fixed effect parameters (i.e., CL, V,
Ka, Ke...).
By some statistical theory, the variation of individual parameter equals
a function of the typical value and the variance of ETA.
VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
If your typical value captures all overall patterns among patients
clearance, then ETA will have a nice symmetric normal distribution with
small variance. Otherwise, you leave too many patterns to ETA and will
see some deviation or shrinkage (whatever you call).
Why adding covariates is a good way to deal with this situation? You
model become CL=typical value*exp (covariate)*exp (ETA). The variation
of individual parameter will be changed to:
VAR (CL) = (typical value + covariate)*exp (omega/2)).
You have one more item to capture the overall patterns, less leave to
ETA. So a 'good' covariate will reduce both the magnitude of omega and
ETA's deviation from normal.
Understanding this is also useful when you are modeling BOV studies.
When you see variation of PK parameters decrease with time (or
occasions). Adding a covariate that make physiological sense and also
decrease with time may help your modeling.
Best,
Xia
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and Rada that says ETABAR can be biased when there is a lot of shrinkage even when the data is simulated and fitted with the correct model. Can you confirm this and can you explain how it arises? In the worst case of shrinkage then bias is impossible because all ETAs must be zero. So why does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas having
> a normal distribution when estimating a parametric model in NONMEM. The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal, but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case: http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
Quoted reply history
> -----Original Message-----
>
> From: BAE, KYUN-SEOP Sent: 13 November 2008 22:05
>
> To: Ribbing, Jakob; XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal, and furthermore non-normal (and/or not-zero-mean) distribution of EBE
>
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
>
> covariate relationships as others suggested, I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> =====================
> Kyun-Seop Bae MD PhD
> Email: [EMAIL PROTECTED]
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: [email protected]
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
>
> significant test. You reject the null hypothesis. More...
>
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter equals
>
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
>
> of individual parameter will be changed to: VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Nick,
The only way I can see ETABAR being biased when fitting the correct
model, is due to asymmetric shrinkage, i.e. that the distribution of EBE
etas is shrunk more in one tail than the other so that the EBE-eta
distribution becomes non-symmetric.
A situation where I would expect this to happen is when putting an "eta
on epsilon" (see ref below). This is a great and simple way of handling
that subjects have different intra-individual error magnitude (instead
of just assuming the same SIGMA for all). In practice, you multiply
whatever the model weight (W) is by e.g. exp(eta) to incorporate eta on
epsilon. This is a simple way of accounting for e.g. that some subjects
are more compliant than others (compliant with therapy, fasting and
other prohibited/compulsory activities during the study).
Assuming that data is not extremely sparse: For subjects where the eta
is highly positive, there will be evidence of them having a higher
variability in the intra-individual error, since their observations
otherwise will become highly unlikely (epsilons which are extremely
positive and negative, in comparison to the value of SIGMA). The eta for
these subject will only be shrunk to a small degree. For the compliant
subject, eps is small (close to zero) for all observations and
consequently, these observations are likely regardless of if the
intra-individual error magnitude is typical or smaller. The eta on these
subjects will shrink from the true (highly negative) eta towards zero.
In consequence, ETABAR can be expected to be positive. This asymmetric
shrinkage does not invalidate the model and it may work great both for
fitting your data and simulate from the model.
Other examples of asymmetric shrinkage may be if there is a continuum of
EC50 values but many subjects where not administered doses high enough
to see a profound effect (all subjects received a low dose so that
drug-effects below Emax have been observed in all): For subjects with
high EC50, that did not receive a high dose, there is no clear effect at
all and the very high eta on EC50 will be shrunk a bit towards zero. For
subjects with low or normal EC50 there will be information in the data
to determine the correct EC50 without shrinkage. The EBE eta
distribution will be skewed to the left, e.g. ranging from -4 to 2, but
still with the median around 0. The model may still be fine, if
alternative parameterisations do not fit the data better.
Best Regards
Jakob
J Pharmacokinet Biopharm. 1995 Dec;23(6):651-72.
Three new residual error models for population PK/PD analyses.Karlsson
MO, Beal SL, Sheiner LB.
Department of Pharmacy, School of Pharmacy, University of California,
San Francisco 94143-0626, USA.
Residual error models, traditionally used in population pharmacokinetic
analyses, have been developed as if all sources of error have properties
similar to those of assay error. Since assay error often is only a minor
part of the difference between predicted and observed concentrations,
other sources, with potentially other properties, should be considered.
We have simulated three complex error structures. The first model
acknowledges two separate sources of residual error, replication error
plus pure residual (assay) error. Simulation results for this case
suggest that ignoring these separate sources of error does not adversely
affect parameter estimates. The second model allows serially correlated
errors, as may occur with structural model misspecification. Ignoring
this error structure leads to biased random-effect parameter estimates.
A simple autocorrelation model, where the correlation between two errors
is assumed to decrease exponentially with the time between them,
provides more accurate estimates of the variability parameters in this
case. The third model allows time-dependent error magnitude. This may be
caused, for example, by inaccurate sample timing. A time-constant error
model fit to time-varying error data can lead to bias in all population
parameter estimates. A simple two-step time-dependent error model is
sufficient to improve parameter estimates, even when the true time
dependence is more complex. Using a real data set, we also illustrate
the use of the different error models to facilitate the model building
process, to provide information about error sources, and to provide more
accurate parameter estimates.
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Nick Holford
Sent: 14 November 2008 00:11
To: nmusers
Subject: Re: [NMusers] Very small P-Value for ETABAR
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and
Rada that says ETABAR can be biased when there is a lot of shrinkage
even when the data is simulated and fitted with the correct model. Can
you confirm this and can you explain how it arises? In the worst case of
shrinkage then bias is impossible because all ETAs must be zero. So why
does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas
having
> a normal distribution when estimating a parametric model in NONMEM.
The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only
when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal,
but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else
reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case:
http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data
using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic
plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
> -----Original Message-----
> From: BAE, KYUN-SEOP
> Sent: 13 November 2008 22:05
> To: Ribbing, Jakob; XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean
of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal,
> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
> covariate relationships as others suggested,
> I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> =====================
> Kyun-Seop Bae MD PhD
> Email: [EMAIL PROTECTED]
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this
is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when
you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to
fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: [email protected]
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates
a
> significant test. You reject the null hypothesis.
>
> More...
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a
lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter
equals
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution
with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
> of individual parameter will be changed to:
>
> VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Nick,
Whenever the amount on information about a parameter is linked to its value
in an individual you can expect asymmetric shrinkage. For example with
standard sampling schemes for PK studies (barring BQL problems), there is
often more information about CL in subjects with high CL than low CL. You
can think of it as have a smaller extrapolated AUC beyond the last
observation. Similarly for example KA will generally be more informed if it
is slow than fast (in individuals with very fast absorption the first
observation may already be beyond the peak). Similarly with EC50 - with a
given concentration range subjects with low EC50 will have a better
characterized profile and more precise estimation of its EC50. As shrinkage
is linked to the information contained in an individual's data shrinkage
will be more pronounced at higher (or lower depending on situation and
parameter) values for individuals and asymmetric shrinkage will result.
Total shrinkage to zero you will only get when you have no individual
information, a situation where you obviously would not have an eta on the
parameter.
I agree with many of the statements. I don't make any decisions the p-values
but would be alerted by large deviations from zero, large being relative to
the magnitude of the variability of the corresponding omega-estimate.
However, even when I see such deviations, I try to check whether it could
come about due to asymmetric shrinkage. The way to do so is first to check
the shrinkage magnitude. If shrinkage is low is low, the deviation is
probably representing a true misfit of the model. If shrinkage is high, then
I may perform a simple simulation from the model parameters, then reestimate
etas using MAXEVAL=0 and check for the ETABAR in this fit. If it is close to
zero, again the high ETABAR in the original model is probably representing a
misfit. If it is of the same size and sign as the ETABAR for the original
data, I would conclude that the high ETABAR in the original fit was a
consequence of asymmetric shrinkage and I wouldn't be disturbed by it.
In my opinion, it would be better if ETABAR had represented the median ETA.
That is always expected to be close to zero for a well-behaved model. A
problem with that is that in the presence of shrinkage, the power to detect
misfit is diminished. Again, it just goes to show that in the presence of
shrinkage, all diagnostics based individual ETAS are less useful.
Best regards,
Mats
Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Nick Holford
Sent: Friday, November 14, 2008 1:11 AM
To: nmusers
Subject: Re: [NMusers] Very small P-Value for ETABAR
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and
Rada that says ETABAR can be biased when there is a lot of shrinkage
even when the data is simulated and fitted with the correct model. Can
you confirm this and can you explain how it arises? In the worst case of
shrinkage then bias is impossible because all ETAs must be zero. So why
does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas having
> a normal distribution when estimating a parametric model in NONMEM. The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal, but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case: http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
> -----Original Message-----
> From: BAE, KYUN-SEOP
> Sent: 13 November 2008 22:05
> To: Ribbing, Jakob; XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal,
> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
> covariate relationships as others suggested,
> I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> =====================
> Kyun-Seop Bae MD PhD
> Email: [EMAIL PROTECTED]
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: [email protected]
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
> significant test. You reject the null hypothesis.
>
> More...
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter equals
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
> of individual parameter will be changed to:
>
> VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Mats, Nick, and NMusers,
When Stu Beal was first thinking about reporting out a p-value for ETABAR I
know he was conflicted because he knew that the statistical properties of
the test were probably never likely to be met. A couple of statistical
properties that are probably not met that haven't been mentioned are:
1) the assumption of independence, and
2) that the individual ETA predictions have constant variance.
The first is not likely to be met because the empirical Bayes predictions of
the ETAs from one individual to the next are correlated because they all
depend on the same set of population parameter estimates. The second is not
met because the precision of the ETA predictions is not constant especially
when there are differences in the number of observations within each
individual.
The p-value, ETABAR, ETA plots, residual plots, COV step output, etc. are
all imperfect diagnostics...often they can be useful but they can also be
misleading. We need to use them cautiously recognizing their limitations and
as Nick has suggested use simulation methods to more fully evaluate our
models.
Best regards,
Ken
Kenneth G. Kowalski
President & CEO
A2PG - Ann Arbor Pharmacometrics Group, Inc.
110 E. Miller Ave., Garden Suite
Ann Arbor, MI 48104
Work: 734-274-8255
Cell: 248-207-5082
Fax: 734-913-0230
[EMAIL PROTECTED]
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Mats Karlsson
Sent: Friday, November 14, 2008 6:39 AM
To: 'Nick Holford'; 'nmusers'
Subject: RE: [NMusers] Very small P-Value for ETABAR
Nick,
Whenever the amount on information about a parameter is linked to its value
in an individual you can expect asymmetric shrinkage. For example with
standard sampling schemes for PK studies (barring BQL problems), there is
often more information about CL in subjects with high CL than low CL. You
can think of it as have a smaller extrapolated AUC beyond the last
observation. Similarly for example KA will generally be more informed if it
is slow than fast (in individuals with very fast absorption the first
observation may already be beyond the peak). Similarly with EC50 - with a
given concentration range subjects with low EC50 will have a better
characterized profile and more precise estimation of its EC50. As shrinkage
is linked to the information contained in an individual's data shrinkage
will be more pronounced at higher (or lower depending on situation and
parameter) values for individuals and asymmetric shrinkage will result.
Total shrinkage to zero you will only get when you have no individual
information, a situation where you obviously would not have an eta on the
parameter.
I agree with many of the statements. I don't make any decisions the p-values
but would be alerted by large deviations from zero, large being relative to
the magnitude of the variability of the corresponding omega-estimate.
However, even when I see such deviations, I try to check whether it could
come about due to asymmetric shrinkage. The way to do so is first to check
the shrinkage magnitude. If shrinkage is low is low, the deviation is
probably representing a true misfit of the model. If shrinkage is high, then
I may perform a simple simulation from the model parameters, then reestimate
etas using MAXEVAL=0 and check for the ETABAR in this fit. If it is close to
zero, again the high ETABAR in the original model is probably representing a
misfit. If it is of the same size and sign as the ETABAR for the original
data, I would conclude that the high ETABAR in the original fit was a
consequence of asymmetric shrinkage and I wouldn't be disturbed by it.
In my opinion, it would be better if ETABAR had represented the median ETA.
That is always expected to be close to zero for a well-behaved model. A
problem with that is that in the presence of shrinkage, the power to detect
misfit is diminished. Again, it just goes to show that in the presence of
shrinkage, all diagnostics based individual ETAS are less useful.
Best regards,
Mats
Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Nick Holford
Sent: Friday, November 14, 2008 1:11 AM
To: nmusers
Subject: Re: [NMusers] Very small P-Value for ETABAR
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and
Rada that says ETABAR can be biased when there is a lot of shrinkage
even when the data is simulated and fitted with the correct model. Can
you confirm this and can you explain how it arises? In the worst case of
shrinkage then bias is impossible because all ETAs must be zero. So why
does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas having
> a normal distribution when estimating a parametric model in NONMEM. The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal, but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case: http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
> -----Original Message-----
> From: BAE, KYUN-SEOP
> Sent: 13 November 2008 22:05
> To: Ribbing, Jakob; XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal,
> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
> covariate relationships as others suggested,
> I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> =====================
> Kyun-Seop Bae MD PhD
> Email: [EMAIL PROTECTED]
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: [email protected]
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
> significant test. You reject the null hypothesis.
>
> More...
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter equals
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
> of individual parameter will be changed to:
>
> VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Hi Jakob,
Thank you very much for the information adding an "eta on epsilon". This is
what I did in my research and I am glad to see people in Pharmacometrics is
using it.
And in Bayesian analysis, adding one more stage for ETA, i.e
ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from zero
and shrinkage of ETA.
Again, thanks all for your input.:)
Best Regards,
Xia
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Ribbing, Jakob
Sent: Friday, November 14, 2008 3:48 AM
To: nmusers
Cc: Nick Holford
Subject: RE: [NMusers] Very small P-Value for ETABAR
Nick,
The only way I can see ETABAR being biased when fitting the correct
model, is due to asymmetric shrinkage, i.e. that the distribution of EBE
etas is shrunk more in one tail than the other so that the EBE-eta
distribution becomes non-symmetric.
A situation where I would expect this to happen is when putting an "eta
on epsilon" (see ref below). This is a great and simple way of handling
that subjects have different intra-individual error magnitude (instead
of just assuming the same SIGMA for all). In practice, you multiply
whatever the model weight (W) is by e.g. exp(eta) to incorporate eta on
epsilon. This is a simple way of accounting for e.g. that some subjects
are more compliant than others (compliant with therapy, fasting and
other prohibited/compulsory activities during the study).
Assuming that data is not extremely sparse: For subjects where the eta
is highly positive, there will be evidence of them having a higher
variability in the intra-individual error, since their observations
otherwise will become highly unlikely (epsilons which are extremely
positive and negative, in comparison to the value of SIGMA). The eta for
these subject will only be shrunk to a small degree. For the compliant
subject, eps is small (close to zero) for all observations and
consequently, these observations are likely regardless of if the
intra-individual error magnitude is typical or smaller. The eta on these
subjects will shrink from the true (highly negative) eta towards zero.
In consequence, ETABAR can be expected to be positive. This asymmetric
shrinkage does not invalidate the model and it may work great both for
fitting your data and simulate from the model.
Other examples of asymmetric shrinkage may be if there is a continuum of
EC50 values but many subjects where not administered doses high enough
to see a profound effect (all subjects received a low dose so that
drug-effects below Emax have been observed in all): For subjects with
high EC50, that did not receive a high dose, there is no clear effect at
all and the very high eta on EC50 will be shrunk a bit towards zero. For
subjects with low or normal EC50 there will be information in the data
to determine the correct EC50 without shrinkage. The EBE eta
distribution will be skewed to the left, e.g. ranging from -4 to 2, but
still with the median around 0. The model may still be fine, if
alternative parameterisations do not fit the data better.
Best Regards
Jakob
J Pharmacokinet Biopharm. 1995 Dec;23(6):651-72.
Three new residual error models for population PK/PD analyses.Karlsson
MO, Beal SL, Sheiner LB.
Department of Pharmacy, School of Pharmacy, University of California,
San Francisco 94143-0626, USA.
Residual error models, traditionally used in population pharmacokinetic
analyses, have been developed as if all sources of error have properties
similar to those of assay error. Since assay error often is only a minor
part of the difference between predicted and observed concentrations,
other sources, with potentially other properties, should be considered.
We have simulated three complex error structures. The first model
acknowledges two separate sources of residual error, replication error
plus pure residual (assay) error. Simulation results for this case
suggest that ignoring these separate sources of error does not adversely
affect parameter estimates. The second model allows serially correlated
errors, as may occur with structural model misspecification. Ignoring
this error structure leads to biased random-effect parameter estimates.
A simple autocorrelation model, where the correlation between two errors
is assumed to decrease exponentially with the time between them,
provides more accurate estimates of the variability parameters in this
case. The third model allows time-dependent error magnitude. This may be
caused, for example, by inaccurate sample timing. A time-constant error
model fit to time-varying error data can lead to bias in all population
parameter estimates. A simple two-step time-dependent error model is
sufficient to improve parameter estimates, even when the true time
dependence is more complex. Using a real data set, we also illustrate
the use of the different error models to facilitate the model building
process, to provide information about error sources, and to provide more
accurate parameter estimates.
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Nick Holford
Sent: 14 November 2008 00:11
To: nmusers
Subject: Re: [NMusers] Very small P-Value for ETABAR
Jakob,
Thanks for some more info on this issue. I have seen work from Mats and
Rada that says ETABAR can be biased when there is a lot of shrinkage
even when the data is simulated and fitted with the correct model. Can
you confirm this and can you explain how it arises? In the worst case of
shrinkage then bias is impossible because all ETAs must be zero. So why
does it occur with non-zero shrinkage?
Nick
Ribbing, Jakob wrote:
> Dear all,
>
> First of all, I am not sure that there is any assumption of etas
having
> a normal distribution when estimating a parametric model in NONMEM.
The
> variance of eta (OMEGA) does not carry an assumption of normality. I
> believe that Stuart used to say the assumption of normality is only
when
> simulating. I guess the assumption also affects EBE:s unless the
> individual information is completely dominating? If the assumption of
> normality is wrong, the weighting of information may not be optimal,
but
> as long as the true distribution is symmetric the estimated parameters
> are in principle correct (but again, the model may not be suitable for
> simulation if the distributional assumption was wrong). I will be off
> line for a few days, but I am sure somebody will correct me if I am
> wrong about this.
>
> If etas are shrunk, you can not expect a normal distribution of that
> (EBE) eta. That does not invalidate parameterization/distributional
> assumptions. Trying other semi-parametric distributions or a
> non-parametric distribution (or a mixture model) may give more
> confidence in sticking with the original parameterization or else
reject
> it as inadequate. In the end, you may feel confident about the model
> even if the EBE eta distribution is asymmetric and biased (I mentioned
> two examples in my earlier posting).
>
> Connecting to how PsN may help in this case:
http://psn.sourceforge.net/
> In practice to evaluate shrinkage, you would simply give the command
> (assuming the model file is called run1.mod):
> execute --shrinkage run1.mod
>
> Another quick evaluation that can be made with this program is to
> produce mirror plots (PsN links in nicely with Xpose for producing the
> diagnostic plots):
>
> execute --mirror=3 run1.mod
>
> This will give you three simulation table files that have been derived
> by simulating under the model and then fitting the simulated data
using
> the same model (using the design of the original data). If you see a
> similar pattern in the mirror plots as in the original diagnostic
plots,
> this gives you more confidence in the model. That brings us back to
> Leonids point about it being more useful to look at diagnostic plots
> than eta bar.
>
> Wishing you a great weekend!
>
> Jakob
>
> -----Original Message-----
> From: BAE, KYUN-SEOP
> Sent: 13 November 2008 22:05
> To: Ribbing, Jakob; XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
> distribution.
> However, the resultant distribution of EBEs may not be normal or mean
of
> them may not be 0.
> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
> But, this practice is discouraged by some (and I agree).
>
> Normal assumption cannot coerce the distribution of EBE to be normal,
> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
> can be nature's nature.
> One simple example is mixture population with polymorphism.
>
> If I could not get normal(?) EBEs even after careful examination of
> covariate relationships as others suggested,
> I would bear it and assume nonparametric distribution.
>
> Regards,
>
> Kyun-Seop
> =====================
> Kyun-Seop Bae MD PhD
> Email: [EMAIL PROTECTED]
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
> On Behalf Of Ribbing, Jakob
> Sent: Thursday, November 13, 2008 13:19
> To: XIA LI; [email protected]
> Subject: RE: [NMusers] Very small P-Value for ETABAR
>
> Hi Xia,
>
> Just to clarify one thing (I agree with almost everything you said):
>
> The p-value indeed is related to the test of ETABAR=0. However, this
is
> not a test of normality, only a test that may reject the mean of the
> etas being zero (H0). Therefore, shrinkage per se does not lead to
> rejection of HO, as long as both tails of the eta distribution are
> shrunk to a similar degree.
>
> I agree with the assumption of normality. This comes into play when
you
> simulate from the model and if you got the distribution of individual
> parameters wrong, simulations may not reflect even the data used to
fit
> the model.
>
> Best Regards
>
> Jakob
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
> On Behalf Of XIA LI
> Sent: 13 November 2008 20:31
> To: [email protected]
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Dear All,
>
> Just some quick statistical points...
>
> P value is usually associated with hypothesis test. As far as I know,
> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
> the null hypothesis to test is H0: ETABAR=0. A small P value indicates
a
> significant test. You reject the null hypothesis.
>
> More...
> As we all know, ETA is used to capture the variation among individual
> parameters and model's unexplained error. We usually use the function
> (or model) parameter=typical value*exp (ETA), which leads to a
lognormal
> distribution assumption for all fixed effect parameters (i.e., CL, V,
> Ka, Ke...).
>
> By some statistical theory, the variation of individual parameter
equals
> a function of the typical value and the variance of ETA.
>
> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
>
> If your typical value captures all overall patterns among patients
> clearance, then ETA will have a nice symmetric normal distribution
with
> small variance. Otherwise, you leave too many patterns to ETA and will
> see some deviation or shrinkage (whatever you call).
>
> Why adding covariates is a good way to deal with this situation? You
> model become CL=typical value*exp (covariate)*exp (ETA). The variation
> of individual parameter will be changed to:
>
> VAR (CL) = (typical value + covariate)*exp (omega/2)).
>
> You have one more item to capture the overall patterns, less leave to
> ETA. So a 'good' covariate will reduce both the magnitude of omega and
> ETA's deviation from normal.
>
> Understanding this is also useful when you are modeling BOV studies.
> When you see variation of PK parameters decrease with time (or
> occasions). Adding a covariate that make physiological sense and also
> decrease with time may help your modeling.
>
> Best,
> Xia
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Xia,
I could be missing something but this
ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
does not make sense to me. In the original definition, ETA(1) is the random variable with normal distribution. Even if posthoc ETAs are not normal, they are still random. For example, it can be either positive or negative (unlike ETA1 given by (1)). If I the understood intentions correctly, this is an attempt to describe a transformation of the random effects to make it normal:
CL = THETA(1) exp(ETA(1)) is replaced by
CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
But not every transformation is reasonable. I hardly can imagine the case when you may want to use (2). Could you give some more realistic examples, please, and situation when they were useful?
On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
Thanks
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Xia Li wrote:
> Hi Nick,
> My pleasure!
>
> This is a topic from Bayesian Hierarchical Model(BHM). If we look at the
> simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the between
> subject random effect. We assume the "similarity" among the subjects may be
> modeled by THETA(1) and ETA(1).
>
> Now here, if we observe that there is an underlying pattern between
> ETA(1)'s, i.e. deviation from zero or no longer normal and we assume that
>
> there is a similarity among those patterns.
>
> Since ETA(1)'s are assumed similar, it is reasonable to model the
> "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
> THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
>
> lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be zero).
>
> We will not say the variance of ETA(1) is confounded with the variance of
> ETA(2), we say it is a function of variance of ETA(2).In statistics,
> confounding means hard to distinguish from each other. Here, it is a direct
> causation.
>
> Sorry I don't have a NM-TRAN code for this now. I usually use SAS and Win
> bugs to do modeling and haven't tried this BHM in NONMEM. I will figure out
> can I do it in NONMEM later.
>
> Best,
> Xia
>
Quoted reply history
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
> Behalf Of Nick Holford
> Sent: Friday, November 14, 2008 3:34 PM
> To: nmusers
> Subject: Re: [NMusers] Very small P-Value for ETABAR
>
> Jakob, Mats,
>
> Thanks very much for your careful explanations of how asymmetric EBE distributions can arise. That is very helpful for my understanding.
>
> Xia,
>
> I am intrigued by your suggestion for how to estimate and account for the bias in the mean of the EBE distribution.
>
> In the usual ETA on EPS model I might write:
>
> ; SD of residual error for mixed proportional and additive random effects
> PROP=THETA(1)*F
> ADD=THETA(2)
> SD=SQRT(PROP*PROP + ADD*ADD)
> Y=F + EPS(1)*SD*EXP(ETA(1))
>
> where EPS(1) is distributed mean zero, variance 1 FIXED
> and ETA(1) is the between subject random effect for residual error
>
> You seem to be suggesting:
> ETABAR=THETA(3)
> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>
> It seems to me that the variance of ETA(1) will be confounded with the variance of ETA(2). Would you please explain more clearly (with an explicit NM-TRAN code fragment if possible) what you are suggesting?
>
> Best wishes,
>
> Nick
>
> Xia Li wrote:
>
> > Hi Jakob,
> > Thank you very much for the information adding an "eta on epsilon". This
>
> is
>
> > what I did in my research and I am glad to see people in Pharmacometrics
>
> is
>
> > using it.
> >
> > And in Bayesian analysis, adding one more stage for ETA, i.e
> > ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from zero
> > and shrinkage of ETA.
> >
> > Again, thanks all for your input.:)
> >
> > Best Regards,
> > Xia
> >
> > Xia Li
> >
> > Mathematical Science Department
> > University of Cincinnati
Leonid,
Sorry, I did make myself clear.
CL=THETA(1)*EXP(ETA(1)) (1)
where ETA(1) is Normal( 0, omega^2) or
log Normal(Eta_bar,omega^2)
Adding one more stage means giving some functions for the MEAN and VARIANCE of
ETA(1), say:
Eta_bar=THETA(2)
omega^= THETA(3)*EXP(ETA(2)) (2)
Sorry for any confusion!
Best,
Xia
Quoted reply history
---- Original message ----
>Date: Fri, 14 Nov 2008 18:37:22 -0500
>From: Leonid Gibiansky <[EMAIL PROTECTED]>
>Subject: Re: [NMusers] Very small P-Value for ETABAR
>To: Xia Li <[EMAIL PROTECTED]>
>Cc: "'Nick Holford'" <[EMAIL PROTECTED]>, "'nmusers'" <[email protected]>
>
>Xia,
>I could be missing something but this
> ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
>does not make sense to me. In the original definition, ETA(1) is the
>random variable with normal distribution. Even if posthoc ETAs are not
>normal, they are still random. For example, it can be either positive or
>negative (unlike ETA1 given by (1)). If I the understood intentions
>correctly, this is an attempt to describe a transformation of the random
>effects to make it normal:
>
>CL = THETA(1) exp(ETA(1)) is replaced by
>CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
>
>But not every transformation is reasonable. I hardly can imagine the
>case when you may want to use (2). Could you give some more realistic
>examples, please, and situation when they were useful?
>
>On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to
>THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
>
>Thanks
>Leonid
>
>--------------------------------------
>Leonid Gibiansky, Ph.D.
>President, QuantPharm LLC
>web: www.quantpharm.com
>e-mail: LGibiansky at quantpharm.com
>tel: (301) 767 5566
>
>
>
>
>Xia Li wrote:
>> Hi Nick,
>> My pleasure!
>>
>> This is a topic from Bayesian Hierarchical Model(BHM). If we look at the
>> simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the between
>> subject random effect. We assume the "similarity" among the subjects may be
>> modeled by THETA(1) and ETA(1).
>>
>> Now here, if we observe that there is an underlying pattern between
>> ETA(1)'s, i.e. deviation from zero or no longer normal and we assume that
>> there is a similarity among those patterns.
>>
>> Since ETA(1)'s are assumed similar, it is reasonable to model the
>> "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
>> THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
>> lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be zero).
>>
>> We will not say the variance of ETA(1) is confounded with the variance of
>> ETA(2), we say it is a function of variance of ETA(2).In statistics,
>> confounding means hard to distinguish from each other. Here, it is a direct
>> causation.
>>
>> Sorry I don't have a NM-TRAN code for this now. I usually use SAS and Win
>> bugs to do modeling and haven't tried this BHM in NONMEM. I will figure out
>> can I do it in NONMEM later.
>>
>> Best,
>> Xia
>>
>> -----Original Message-----
>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
>> Behalf Of Nick Holford
>> Sent: Friday, November 14, 2008 3:34 PM
>> To: nmusers
>> Subject: Re: [NMusers] Very small P-Value for ETABAR
>>
>> Jakob, Mats,
>>
>> Thanks very much for your careful explanations of how asymmetric EBE
>> distributions can arise. That is very helpful for my understanding.
>>
>> Xia,
>>
>> I am intrigued by your suggestion for how to estimate and account for
>> the bias in the mean of the EBE distribution.
>>
>> In the usual ETA on EPS model I might write:
>>
>> ; SD of residual error for mixed proportional and additive random effects
>> PROP=THETA(1)*F
>> ADD=THETA(2)
>> SD=SQRT(PROP*PROP + ADD*ADD)
>> Y=F + EPS(1)*SD*EXP(ETA(1))
>>
>> where EPS(1) is distributed mean zero, variance 1 FIXED
>> and ETA(1) is the between subject random effect for residual error
>>
>> You seem to be suggesting:
>> ETABAR=THETA(3)
>> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>>
>> It seems to me that the variance of ETA(1) will be confounded with the
>> variance of ETA(2). Would you please explain more clearly (with an
>> explicit NM-TRAN code fragment if possible) what you are suggesting?
>>
>> Best wishes,
>>
>> Nick
>>
>> Xia Li wrote:
>>> Hi Jakob,
>>> Thank you very much for the information adding an "eta on epsilon". This
>> is
>>> what I did in my research and I am glad to see people in Pharmacometrics
>> is
>>> using it.
>>>
>>> And in Bayesian analysis, adding one more stage for ETA, i.e
>>> ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from zero
>>> and shrinkage of ETA.
>>>
>>> Again, thanks all for your input.:)
>>>
>>> Best Regards,
>>> Xia
>>>
>>> Xia Li
>>> Mathematical Science Department
>>> University of Cincinnati
>>>
>>
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
Xia,
I wrote:
> ETABAR=THETA(3)
> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>
> It seems to me that the variance of ETA(1) will be confounded with the variance of ETA(2). Would you please explain more clearly (with an explicit NM-TRAN code fragment if possible) what you are suggesting?
Leonid added:
> CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
>
> But not every transformation is reasonable. I hardly can imagine the case when you may want to use (2). Could you give some more realistic examples, please, and situation when they were useful?
You replied but between
"Sorry, I did make myself clear."
and
"Sorry for any confusion!"
I only found unclear and confusing remarks (e.g. where is ETABAR actually used?)
Would you please focus more on answering our specific requests for an explicit
NM-TRAN code fragment and justification for an apparently bizarre
transformation and spend less time offering meaningless apologies?
Nick
XIA LI wrote:
> Leonid,
>
> Sorry, I did make myself clear.
>
> CL=THETA(1)*EXP(ETA(1)) (1)
>
> where ETA(1) is Normal( 0, omega^2) or log Normal(Eta_bar,omega^2)
>
> Adding one more stage means giving some functions for the MEAN and VARIANCE of
> ETA(1), say:
>
> Eta_bar=THETA(2)
> omega^= THETA(3)*EXP(ETA(2)) (2)
>
> Sorry for any confusion!
> Best,
> Xia
>
Quoted reply history
> ---- Original message ----
>
> > Date: Fri, 14 Nov 2008 18:37:22 -0500
> >
> > From: Leonid Gibiansky <[EMAIL PROTECTED]> Subject: Re: [NMusers] Very small P-Value for ETABAR To: Xia Li <[EMAIL PROTECTED]>
> >
> > Cc: "'Nick Holford'" <[EMAIL PROTECTED]>, "'nmusers'" <[email protected]>
> >
> > Xia,
> > I could be missing something but this
> > ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
> >
> > does not make sense to me. In the original definition, ETA(1) is the random variable with normal distribution. Even if posthoc ETAs are not normal, they are still random. For example, it can be either positive or negative (unlike ETA1 given by (1)). If I the understood intentions correctly, this is an attempt to describe a transformation of the random effects to make it normal:
> >
> > CL = THETA(1) exp(ETA(1)) is replaced by
> > CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
> >
> > But not every transformation is reasonable. I hardly can imagine the case when you may want to use (2). Could you give some more realistic examples, please, and situation when they were useful?
> >
> > On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
> >
> > Thanks
> > Leonid
> >
> > --------------------------------------
> > Leonid Gibiansky, Ph.D.
> > President, QuantPharm LLC
> > web: www.quantpharm.com
> > e-mail: LGibiansky at quantpharm.com
> > tel: (301) 767 5566
> >
> > Xia Li wrote:
> >
> > > Hi Nick,
> > > My pleasure!
> > >
> > > This is a topic from Bayesian Hierarchical Model(BHM). If we look at the
> > > simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the between
> > > subject random effect. We assume the "similarity" among the subjects may be
> > > modeled by THETA(1) and ETA(1).
> > >
> > > Now here, if we observe that there is an underlying pattern between
> > > ETA(1)'s, i.e. deviation from zero or no longer normal and we assume that
> > >
> > > there is a similarity among those patterns.
> > >
> > > Since ETA(1)'s are assumed similar, it is reasonable to model the
> > > "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
> > > THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
> > >
> > > lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be zero).
> > >
> > > We will not say the variance of ETA(1) is confounded with the variance of
> > > ETA(2), we say it is a function of variance of ETA(2).In statistics,
> > > confounding means hard to distinguish from each other. Here, it is a direct
> > > causation.
> > >
> > > Sorry I don't have a NM-TRAN code for this now. I usually use SAS and Win
> > > bugs to do modeling and haven't tried this BHM in NONMEM. I will figure out
> > > can I do it in NONMEM later.
> > >
> > > Best,
> > > Xia
> > >
> > > -----Original Message-----
> > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
> > > Behalf Of Nick Holford
> > > Sent: Friday, November 14, 2008 3:34 PM
> > > To: nmusers
> > > Subject: Re: [NMusers] Very small P-Value for ETABAR
> > >
> > > Jakob, Mats,
> > >
> > > Thanks very much for your careful explanations of how asymmetric EBE distributions can arise. That is very helpful for my understanding.
> > >
> > > Xia,
> > >
> > > I am intrigued by your suggestion for how to estimate and account for the bias in the mean of the EBE distribution.
> > >
> > > In the usual ETA on EPS model I might write:
> > >
> > > ; SD of residual error for mixed proportional and additive random effects
> > > PROP=THETA(1)*F
> > > ADD=THETA(2)
> > > SD=SQRT(PROP*PROP + ADD*ADD)
> > > Y=F + EPS(1)*SD*EXP(ETA(1))
> > >
> > > where EPS(1) is distributed mean zero, variance 1 FIXED
> > > and ETA(1) is the between subject random effect for residual error
> > >
> > > You seem to be suggesting:
> > > ETABAR=THETA(3)
> > > Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
> > >
> > > It seems to me that the variance of ETA(1) will be confounded with the variance of ETA(2). Would you please explain more clearly (with an explicit NM-TRAN code fragment if possible) what you are suggesting?
> > >
> > > Best wishes,
> > >
> > > Nick
> > >
> > > Xia Li wrote:
> > >
> > > > Hi Jakob,
> > > > Thank you very much for the information adding an "eta on epsilon". This
> > >
> > > is
> > >
> > > > what I did in my research and I am glad to see people in Pharmacometrics
> > >
> > > is
> > >
> > > > using it.
> > > >
> > > > And in Bayesian analysis, adding one more stage for ETA, i.e
> > > > ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from zero
> > > > and shrinkage of ETA.
> > > >
> > > > Again, thanks all for your input.:)
> > > >
> > > > Best Regards,
> > > > Xia
> > > >
> > > > Xia Li
> > > >
> > > > Mathematical Science Department
> > > > University of Cincinnati
>
> ======================================
> Xia Li
> Mathematical Science Department
> University of Cincinnati
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Xia,
I must admit, I am still confused. In my mind, you can not estimate
THETA(2) in your code, since it is completely confounded with THETA(1).
Moreover, if you fix THETA(2) to a non-zero value, THETA(1) will no
longer be the typical value of CL (or the population typical value of
CL), meaning that the interpretability of THETA(1) is lost.
Regarding your definition of omega^ I think this is an attempt to allow
a semi-parametric model. Can you please explain how equation 2 affects
equation 1, using code acceptable in the nonmem NONMEM program?
Currently, I am not clear on how many random effects you are estimating
for CL.
Thanks
Jakob
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of XIA LI
Sent: 17 November 2008 05:28
To: Leonid Gibiansky
Cc: 'Nick Holford'; 'nmusers'
Subject: Re: [NMusers] Very small P-Value for ETABAR
Leonid,
Sorry, I did make myself clear.
CL=THETA(1)*EXP(ETA(1)) (1)
where ETA(1) is Normal( 0, omega^2) or
log Normal(Eta_bar,omega^2)
Adding one more stage means giving some functions for the MEAN and
VARIANCE of ETA(1), say:
Eta_bar=THETA(2)
omega^= THETA(3)*EXP(ETA(2)) (2)
Sorry for any confusion!
Best,
Xia
---- Original message ----
>Date: Fri, 14 Nov 2008 18:37:22 -0500
>From: Leonid Gibiansky <[EMAIL PROTECTED]>
>Subject: Re: [NMusers] Very small P-Value for ETABAR
>To: Xia Li <[EMAIL PROTECTED]>
>Cc: "'Nick Holford'" <[EMAIL PROTECTED]>, "'nmusers'"
<[email protected]>
>
>Xia,
>I could be missing something but this
> ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
>does not make sense to me. In the original definition, ETA(1) is the
>random variable with normal distribution. Even if posthoc ETAs are not
>normal, they are still random. For example, it can be either positive
or
>negative (unlike ETA1 given by (1)). If I the understood intentions
>correctly, this is an attempt to describe a transformation of the
random
>effects to make it normal:
>
>CL = THETA(1) exp(ETA(1)) is replaced by
>CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
>
>But not every transformation is reasonable. I hardly can imagine the
>case when you may want to use (2). Could you give some more realistic
>examples, please, and situation when they were useful?
>
>On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to
>THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
>
>Thanks
>Leonid
>
>--------------------------------------
>Leonid Gibiansky, Ph.D.
>President, QuantPharm LLC
>web: www.quantpharm.com
>e-mail: LGibiansky at quantpharm.com
>tel: (301) 767 5566
>
>
>
>
>Xia Li wrote:
>> Hi Nick,
>> My pleasure!
>>
>> This is a topic from Bayesian Hierarchical Model(BHM). If we look at
the
>> simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the
between
>> subject random effect. We assume the "similarity" among the subjects
may be
>> modeled by THETA(1) and ETA(1).
>>
>> Now here, if we observe that there is an underlying pattern between
>> ETA(1)'s, i.e. deviation from zero or no longer normal and we assume
that
>> there is a similarity among those patterns.
>>
>> Since ETA(1)'s are assumed similar, it is reasonable to model the
>> "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
>> THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
>> lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be
zero).
>>
>> We will not say the variance of ETA(1) is confounded with the
variance of
>> ETA(2), we say it is a function of variance of ETA(2).In statistics,
>> confounding means hard to distinguish from each other. Here, it is a
direct
>> causation.
>>
>> Sorry I don't have a NM-TRAN code for this now. I usually use SAS and
Win
>> bugs to do modeling and haven't tried this BHM in NONMEM. I will
figure out
>> can I do it in NONMEM later.
>>
>> Best,
>> Xia
>>
>> -----Original Message-----
>> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
>> Behalf Of Nick Holford
>> Sent: Friday, November 14, 2008 3:34 PM
>> To: nmusers
>> Subject: Re: [NMusers] Very small P-Value for ETABAR
>>
>> Jakob, Mats,
>>
>> Thanks very much for your careful explanations of how asymmetric EBE
>> distributions can arise. That is very helpful for my understanding.
>>
>> Xia,
>>
>> I am intrigued by your suggestion for how to estimate and account for
>> the bias in the mean of the EBE distribution.
>>
>> In the usual ETA on EPS model I might write:
>>
>> ; SD of residual error for mixed proportional and additive random
effects
>> PROP=THETA(1)*F
>> ADD=THETA(2)
>> SD=SQRT(PROP*PROP + ADD*ADD)
>> Y=F + EPS(1)*SD*EXP(ETA(1))
>>
>> where EPS(1) is distributed mean zero, variance 1 FIXED
>> and ETA(1) is the between subject random effect for residual error
>>
>> You seem to be suggesting:
>> ETABAR=THETA(3)
>> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>>
>> It seems to me that the variance of ETA(1) will be confounded with
the
>> variance of ETA(2). Would you please explain more clearly (with an
>> explicit NM-TRAN code fragment if possible) what you are suggesting?
>>
>> Best wishes,
>>
>> Nick
>>
>> Xia Li wrote:
>>> Hi Jakob,
>>> Thank you very much for the information adding an "eta on epsilon".
This
>> is
>>> what I did in my research and I am glad to see people in
Pharmacometrics
>> is
>>> using it.
>>>
>>> And in Bayesian analysis, adding one more stage for ETA, i.e
>>> ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from
zero
>>> and shrinkage of ETA.
>>>
>>> Again, thanks all for your input.:)
>>>
>>> Best Regards,
>>> Xia
>>>
>>> Xia Li
>>> Mathematical Science Department
>>> University of Cincinnati
>>>
>>
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
Hello all,
This is my understanding of the issue on the influence of non-normal etas
and the quality of the prediction of these, such as measured through
shrinkage, on the estimation of model parameters in NONMEM.
NONMEM uses the extended least squares (ELS) procedure to estimate the fixed
(thetas) and variance components of the random effects (omegas) jointly. If
the data between individuals are independent and normally distributed and
the etas enter the model linearly then ELS estimation is equivalent to
maximum likelihood estimation. This implies the estimates should be
consistent, asymptotically normally distributed, and asymptotically
efficient (small standard errors) - based on sufficient sample size. If the
data are not normally distributed, the ELS estimates are still consistent
and asymptotically normally distributed, but lose some efficiency
(relatively larger standard errors of the estimates). Estimation is no
longer considered maximum likelihood estimation. It is my understanding
that Sheiner and Beal coined the term extended least squares because of the
nice property for non-normal data - that the estimates are still consistent
(unbiased for large samples sizes). So for models linear in the etas, it is
true that the distribution of population residuals (based on the etas and
epsilons) should not adversely affect estimation as long as the mean and
marginal (or population) variance (based on the omegas and sigmas) are
correctly specified. The sandwich estimator of the standard errors,
referred to in NONMEM as R^-1*S*R^-1 ("^" is the exponential operator)
ensures consistent estimates of the standard errors for thetas, omegas, and
epsilons in the case of non-normal data (essentially it is robust to the
distributional assumptions of the data).
When the etas enter the model nonlinearly, things get more complicated in
NONMEM. The etas are assumed to be normally distributed to facilitate a
convenient approximation (FO or the Laplacian based FOCE, etc) to the
marginal likelihood. This approximation appears as a multivariate normal
distribution. This allows the use of ELS to estimate the parameters. Thus,
the assumption of normality of the etas directly relates to the
approximation implemented to estimate the parameters. What happens if the
eta's are not normal is not directly clear with respect to the approximation
and hence the estimates. Also, if the distribution of the etas is markedly
skewed, the interpretation of the model prediction with etas=0 as the
"typical individual" model prediction is probably no longer appropriate.
This is because the prediction at eta=0 is no longer at the most likely eta
value (0 is most likely in symmetric distributions). These things are
avoided in the linear case above because the linear model is parameterized
directly with respect to the population mean, so the thetas in that model
are already interpretable with respect to the population of the data despite
the lack of normality.
So, when the etas are nonlinear in the model, the FO or FOCE model is now an
approximate model in that the means and marginal variances are only
approximately correct. For FOCE, how close these are to 'correct' depends
upon how good the etas are predicted, which is a function of the amount of
data within an individual (i.e. data quality), and because the theta's and
omega's apply to all individuals, criteria for sufficient data within all
individuals also need be met. This result is related to the FOCE versus FO
issue with bias (as data become sparse FOCE approaches FO - perhaps
quantified conveniently by shrinkage) and the WRES versus CWRES for
residuals (CWRES go to WRES as data become sparse). But as stated, we are
fitting the likelihood to an approximate model and so bias can result
because of the approximation, ie the (approximately) incorrect mean and
variance (which is a function of the etas and the distributional assumption
of these). This relates to the quality of the etas.
However, we should not get too depressed because the Laplace method works
well quite often for estimation, and in my experience, the first place it
tends to fail is with respect to estimating the omegas (compared to better
approximations like adaptive Gaussian quadrature).
References:
NONMEM Users Guides.
LB Sheiner, SL BeaL. Pharmacokinetic parameter estimates from several least
squares procedures: superiority of extended least squares. J Pharmacokinet
Biopharm. 1985 Apr;13(2):185-201.
CC Peck, SL Beal, LB Sheiner AI Nichols. Extended least squares nonlinear
regression: A possible solution to the "choice of weights" problem in
analysis of individual pharmacokinetic data , JPP Volume 12, Number 5 /
October, 1984
SL Beal. Commentary on Significance Levels for Covariate Effects in NONMEM
JPP Volume 29, Number 4 / August, 2002
Vonesh and chinchilli. Linear and Nonlinear models for the analysis of
repeated measurements. Marcel Dekker.
EF Vonesh. A note on the use of Laplace's approximation for nonlinear
mixed-effects models. Biometrika, June 1996; 83: 447 - 452.
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Ribbing, Jakob
Sent: Thursday, November 13, 2008 6:28 PM
To: [email protected]
Cc: BAE, KYUN-SEOP; XIA LI
Subject: RE: [NMusers] Very small P-Value for ETABAR
Dear all,
First of all, I am not sure that there is any assumption of etas having
a normal distribution when estimating a parametric model in NONMEM. The
variance of eta (OMEGA) does not carry an assumption of normality. I
believe that Stuart used to say the assumption of normality is only when
simulating. I guess the assumption also affects EBE:s unless the
individual information is completely dominating? If the assumption of
normality is wrong, the weighting of information may not be optimal, but
as long as the true distribution is symmetric the estimated parameters
are in principle correct (but again, the model may not be suitable for
simulation if the distributional assumption was wrong). I will be off
line for a few days, but I am sure somebody will correct me if I am
wrong about this.
If etas are shrunk, you can not expect a normal distribution of that
(EBE) eta. That does not invalidate parameterization/distributional
assumptions. Trying other semi-parametric distributions or a
non-parametric distribution (or a mixture model) may give more
confidence in sticking with the original parameterization or else reject
it as inadequate. In the end, you may feel confident about the model
even if the EBE eta distribution is asymmetric and biased (I mentioned
two examples in my earlier posting).
Connecting to how PsN may help in this case: http://psn.sourceforge.net/
In practice to evaluate shrinkage, you would simply give the command
(assuming the model file is called run1.mod):
execute --shrinkage run1.mod
Another quick evaluation that can be made with this program is to
produce mirror plots (PsN links in nicely with Xpose for producing the
diagnostic plots):
execute --mirror=3 run1.mod
This will give you three simulation table files that have been derived
by simulating under the model and then fitting the simulated data using
the same model (using the design of the original data). If you see a
similar pattern in the mirror plots as in the original diagnostic plots,
this gives you more confidence in the model. That brings us back to
Leonids point about it being more useful to look at diagnostic plots
than eta bar.
Wishing you a great weekend!
Jakob
-----Original Message-----
From: BAE, KYUN-SEOP
Sent: 13 November 2008 22:05
To: Ribbing, Jakob; XIA LI; [email protected]
Subject: RE: [NMusers] Very small P-Value for ETABAR
Dear All,
Realized etas (EBEs, MAPs) is estimated under the assumption of normal
distribution.
However, the resultant distribution of EBEs may not be normal or mean of
them may not be 0.
To pass t-test, one may use "CENTERING" option at $ESTIMATION.
But, this practice is discouraged by some (and I agree).
Normal assumption cannot coerce the distribution of EBE to be normal,
and furthermore non-normal (and/or not-zero-mean) distribution of EBE
can be nature's nature.
One simple example is mixture population with polymorphism.
If I could not get normal(?) EBEs even after careful examination of
covariate relationships as others suggested,
I would bear it and assume nonparametric distribution.
Regards,
Kyun-Seop
=====================
Kyun-Seop Bae MD PhD
Email: [EMAIL PROTECTED]
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Ribbing, Jakob
Sent: Thursday, November 13, 2008 13:19
To: XIA LI; [email protected]
Subject: RE: [NMusers] Very small P-Value for ETABAR
Hi Xia,
Just to clarify one thing (I agree with almost everything you said):
The p-value indeed is related to the test of ETABAR=0. However, this is
not a test of normality, only a test that may reject the mean of the
etas being zero (H0). Therefore, shrinkage per se does not lead to
rejection of HO, as long as both tails of the eta distribution are
shrunk to a similar degree.
I agree with the assumption of normality. This comes into play when you
simulate from the model and if you got the distribution of individual
parameters wrong, simulations may not reflect even the data used to fit
the model.
Best Regards
Jakob
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of XIA LI
Sent: 13 November 2008 20:31
To: [email protected]
Subject: Re: [NMusers] Very small P-Value for ETABAR
Dear All,
Just some quick statistical points...
P value is usually associated with hypothesis test. As far as I know,
NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
significant test. You reject the null hypothesis.
More...
As we all know, ETA is used to capture the variation among individual
parameters and model's unexplained error. We usually use the function
(or model) parameter=typical value*exp (ETA), which leads to a lognormal
distribution assumption for all fixed effect parameters (i.e., CL, V,
Ka, Ke...).
By some statistical theory, the variation of individual parameter equals
a function of the typical value and the variance of ETA.
VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
If your typical value captures all overall patterns among patients
clearance, then ETA will have a nice symmetric normal distribution with
small variance. Otherwise, you leave too many patterns to ETA and will
see some deviation or shrinkage (whatever you call).
Why adding covariates is a good way to deal with this situation? You
model become CL=typical value*exp (covariate)*exp (ETA). The variation
of individual parameter will be changed to:
VAR (CL) = (typical value + covariate)*exp (omega/2)).
You have one more item to capture the overall patterns, less leave to
ETA. So a 'good' covariate will reduce both the magnitude of omega and
ETA's deviation from normal.
Understanding this is also useful when you are modeling BOV studies.
When you see variation of PK parameters decrease with time (or
occasions). Adding a covariate that make physiological sense and also
decrease with time may help your modeling.
Best,
Xia
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
I just get back from class and will try to answer the question.
Best,
Xia
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Ribbing, Jakob
Sent: Monday, November 17, 2008 2:27 AM
To: XIA LI; nmusers
Subject: RE: [NMusers] Very small P-Value for ETABAR
Xia,
I must admit, I am still confused. In my mind, you can not estimate
THETA(2) in your code, since it is completely confounded with THETA(1).
Moreover, if you fix THETA(2) to a non-zero value, THETA(1) will no
longer be the typical value of CL (or the population typical value of
CL), meaning that the interpretability of THETA(1) is lost.
Regarding your definition of omega^ I think this is an attempt to allow
a semi-parametric model. Can you please explain how equation 2 affects
equation 1, using code acceptable in the nonmem NONMEM program?
Currently, I am not clear on how many random effects you are estimating
for CL.
Thanks
Jakob
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of XIA LI
Sent: 17 November 2008 05:28
To: Leonid Gibiansky
Cc: 'Nick Holford'; 'nmusers'
Subject: Re: [NMusers] Very small P-Value for ETABAR
Leonid,
Sorry, I did make myself clear.
CL=THETA(1)*EXP(ETA(1)) (1)
where ETA(1) is Normal( 0, omega^2) or
log Normal(Eta_bar,omega^2)
Adding one more stage means giving some functions for the MEAN and
VARIANCE of ETA(1), say:
Eta_bar=THETA(2)
omega^= THETA(3)*EXP(ETA(2)) (2)
Sorry for any confusion!
Best,
Xia
---- Original message ----
>Date: Fri, 14 Nov 2008 18:37:22 -0500
>From: Leonid Gibiansky <[EMAIL PROTECTED]>
>Subject: Re: [NMusers] Very small P-Value for ETABAR
>To: Xia Li <[EMAIL PROTECTED]>
>Cc: "'Nick Holford'" <[EMAIL PROTECTED]>, "'nmusers'"
<[email protected]>
>
>Xia,
>I could be missing something but this
> ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
>does not make sense to me. In the original definition, ETA(1) is the
>random variable with normal distribution. Even if posthoc ETAs are not
>normal, they are still random. For example, it can be either positive
or
>negative (unlike ETA1 given by (1)). If I the understood intentions
>correctly, this is an attempt to describe a transformation of the
random
>effects to make it normal:
>
>CL = THETA(1) exp(ETA(1)) is replaced by
>CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
>
>But not every transformation is reasonable. I hardly can imagine the
>case when you may want to use (2). Could you give some more realistic
>examples, please, and situation when they were useful?
>
>On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to
>THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
>
>Thanks
>Leonid
>
>--------------------------------------
>Leonid Gibiansky, Ph.D.
>President, QuantPharm LLC
>web: www.quantpharm.com
>e-mail: LGibiansky at quantpharm.com
>tel: (301) 767 5566
>
>
>
>
>Xia Li wrote:
>> Hi Nick,
>> My pleasure!
>>
>> This is a topic from Bayesian Hierarchical Model(BHM). If we look at
the
>> simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the
between
>> subject random effect. We assume the "similarity" among the subjects
may be
>> modeled by THETA(1) and ETA(1).
>>
>> Now here, if we observe that there is an underlying pattern between
>> ETA(1)'s, i.e. deviation from zero or no longer normal and we assume
that
>> there is a similarity among those patterns.
>>
>> Since ETA(1)'s are assumed similar, it is reasonable to model the
>> "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
>> THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
>> lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be
zero).
>>
>> We will not say the variance of ETA(1) is confounded with the
variance of
>> ETA(2), we say it is a function of variance of ETA(2).In statistics,
>> confounding means hard to distinguish from each other. Here, it is a
direct
>> causation.
>>
>> Sorry I don't have a NM-TRAN code for this now. I usually use SAS and
Win
>> bugs to do modeling and haven't tried this BHM in NONMEM. I will
figure out
>> can I do it in NONMEM later.
>>
>> Best,
>> Xia
>>
>> -----Original Message-----
>> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
>> Behalf Of Nick Holford
>> Sent: Friday, November 14, 2008 3:34 PM
>> To: nmusers
>> Subject: Re: [NMusers] Very small P-Value for ETABAR
>>
>> Jakob, Mats,
>>
>> Thanks very much for your careful explanations of how asymmetric EBE
>> distributions can arise. That is very helpful for my understanding.
>>
>> Xia,
>>
>> I am intrigued by your suggestion for how to estimate and account for
>> the bias in the mean of the EBE distribution.
>>
>> In the usual ETA on EPS model I might write:
>>
>> ; SD of residual error for mixed proportional and additive random
effects
>> PROP=THETA(1)*F
>> ADD=THETA(2)
>> SD=SQRT(PROP*PROP + ADD*ADD)
>> Y=F + EPS(1)*SD*EXP(ETA(1))
>>
>> where EPS(1) is distributed mean zero, variance 1 FIXED
>> and ETA(1) is the between subject random effect for residual error
>>
>> You seem to be suggesting:
>> ETABAR=THETA(3)
>> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>>
>> It seems to me that the variance of ETA(1) will be confounded with
the
>> variance of ETA(2). Would you please explain more clearly (with an
>> explicit NM-TRAN code fragment if possible) what you are suggesting?
>>
>> Best wishes,
>>
>> Nick
>>
>> Xia Li wrote:
>>> Hi Jakob,
>>> Thank you very much for the information adding an "eta on epsilon".
This
>> is
>>> what I did in my research and I am glad to see people in
Pharmacometrics
>> is
>>> using it.
>>>
>>> And in Bayesian analysis, adding one more stage for ETA, i.e
>>> ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from
zero
>>> and shrinkage of ETA.
>>>
>>> Again, thanks all for your input.:)
>>>
>>> Best Regards,
>>> Xia
>>>
>>> Xia Li
>>> Mathematical Science Department
>>> University of Cincinnati
>>>
>>
======================================
Xia Li
Mathematical Science Department
University of Cincinnati
Dear All,
We begin with
CL=THETA(1)*EXP(ETA(1)), (1)
where ETA(1) is N( 0, omega^2),
So the p-value for testing mean of ETA(1)=0 would be "expected" to be
large.
When (1) doesn't look like the right model, say b/c mean of ETA(1) looks
non-zero,
I said: modify (1) by letting
ETA(1)= THETA(2)*exp(ETA(2)) (2a).
with ETA(2) being N(0.v), so that ETA(1) now has possibly non-zero mean.
Jakob you are right, combing (1) and (2a) we see THETA(1) and THETA(2)
appear together making both THETA's non-identifiable.
What I probably thought was, instead of (2a), set
EXP(ETA(1))= exp(W)*exp(ETA(2))= exp(W+ETA(2)) (2b)
for a covariate W. Combining (1) and (2b)
CL=THETA(1)*EXP(ETA(1))= THETA(1)*exp(W)*exp(ETA(2))
giving mean of ETA(1) non-zero, and dependent on covariate W. We are back to
the point that 'good' covariates help modeling...
Regarding the question how equation 2c affects equation 1 in my previous
email.
omega^2= THETA(3)*EXP(ETA(2)) (2c)
Instead of assuming ETA(1) is normally distributed with same variance
omega^2, we say different omega^2 for different characteristic groups.
Omega^2_j=theta3_j*exp(eta2_ij)
By doing so, ETA(1) is a mixture of j normals with different variance. We
know a mixture of normals (having identical mean and variance) is normal,
whereas, a mixture of normals (if means are identical, but variances are
not) may be non-normal.
Now, 2(b) may help explaining the nonzero mean of ETA(1) and 2(c) may help
explaining the asymmetric shape of ETA(1).
I tried to attach a doodles graph to help my explanation but it seems
attachment is not allowed...
http://i36.tinypic.com/152hpwy.jpg
Again, thanks for all input and let me know if I missed something or
misunderstood the original problem.
Best,
Xia
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Ribbing, Jakob
Sent: Monday, November 17, 2008 2:27 AM
To: XIA LI; nmusers
Subject: RE: [NMusers] Very small P-Value for ETABAR
Xia,
I must admit, I am still confused. In my mind, you can not estimate
THETA(2) in your code, since it is completely confounded with THETA(1).
Moreover, if you fix THETA(2) to a non-zero value, THETA(1) will no
longer be the typical value of CL (or the population typical value of
CL), meaning that the interpretability of THETA(1) is lost.
Regarding your definition of omega^ I think this is an attempt to allow
a semi-parametric model. Can you please explain how equation 2 affects
equation 1, using code acceptable in the nonmem NONMEM program?
Currently, I am not clear on how many random effects you are estimating
for CL.
Thanks
Jakob
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of XIA LI
Sent: 17 November 2008 05:28
To: Leonid Gibiansky
Cc: 'Nick Holford'; 'nmusers'
Subject: Re: [NMusers] Very small P-Value for ETABAR
Leonid,
Sorry, I did make myself clear.
CL=THETA(1)*EXP(ETA(1)) (1)
where ETA(1) is Normal( 0, omega^2) or
log Normal(Eta_bar,omega^2)
Adding one more stage means giving some functions for the MEAN and
VARIANCE of ETA(1), say:
Eta_bar=THETA(2)
omega^= THETA(3)*EXP(ETA(2)) (2)
Sorry for any confusion!
Best,
Xia
---- Original message ----
>Date: Fri, 14 Nov 2008 18:37:22 -0500
>From: Leonid Gibiansky <[EMAIL PROTECTED]>
>Subject: Re: [NMusers] Very small P-Value for ETABAR
>To: Xia Li <[EMAIL PROTECTED]>
>Cc: "'Nick Holford'" <[EMAIL PROTECTED]>, "'nmusers'"
<[email protected]>
>
>Xia,
>I could be missing something but this
> ETA(1)= THETA(2)*exp(ETA(2)) (Eq. 1)
>does not make sense to me. In the original definition, ETA(1) is the
>random variable with normal distribution. Even if posthoc ETAs are not
>normal, they are still random. For example, it can be either positive
or
>negative (unlike ETA1 given by (1)). If I the understood intentions
>correctly, this is an attempt to describe a transformation of the
random
>effects to make it normal:
>
>CL = THETA(1) exp(ETA(1)) is replaced by
>CL = THETA(1) exp(THETA(2)*exp(ETA(1))) (2)
>
>But not every transformation is reasonable. I hardly can imagine the
>case when you may want to use (2). Could you give some more realistic
>examples, please, and situation when they were useful?
>
>On the separate note, mean of THETA(2)*exp(ETA(2)) is not equal to
>THETA(2): geometric mean of THETA(2)*exp(ETA(2)) is equal to THETA(2)
>
>Thanks
>Leonid
>
>--------------------------------------
>Leonid Gibiansky, Ph.D.
>President, QuantPharm LLC
>web: www.quantpharm.com
>e-mail: LGibiansky at quantpharm.com
>tel: (301) 767 5566
>
>
>
>
>Xia Li wrote:
>> Hi Nick,
>> My pleasure!
>>
>> This is a topic from Bayesian Hierarchical Model(BHM). If we look at
the
>> simplest PK statement: CL=THETA(1)*EXP(ETA(1)), where ETA(1) is the
between
>> subject random effect. We assume the "similarity" among the subjects
may be
>> modeled by THETA(1) and ETA(1).
>>
>> Now here, if we observe that there is an underlying pattern between
>> ETA(1)'s, i.e. deviation from zero or no longer normal and we assume
that
>> there is a similarity among those patterns.
>>
>> Since ETA(1)'s are assumed similar, it is reasonable to model the
>> "similarity" among the ETA(1)'s by THETA(2) and ETA(2): ETA(1)=
>> THETA(2)*exp(ETA(2)). Hence we have one more stage, ETA(1) now is
>> lognormal(nonsymmetrical) with mean THETA(2) (doesnt have to be
zero).
>>
>> We will not say the variance of ETA(1) is confounded with the
variance of
>> ETA(2), we say it is a function of variance of ETA(2).In statistics,
>> confounding means hard to distinguish from each other. Here, it is a
direct
>> causation.
>>
>> Sorry I don't have a NM-TRAN code for this now. I usually use SAS and
Win
>> bugs to do modeling and haven't tried this BHM in NONMEM. I will
figure out
>> can I do it in NONMEM later.
>>
>> Best,
>> Xia
>>
>> -----Original Message-----
>> From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
>> Behalf Of Nick Holford
>> Sent: Friday, November 14, 2008 3:34 PM
>> To: nmusers
>> Subject: Re: [NMusers] Very small P-Value for ETABAR
>>
>> Jakob, Mats,
>>
>> Thanks very much for your careful explanations of how asymmetric EBE
>> distributions can arise. That is very helpful for my understanding.
>>
>> Xia,
>>
>> I am intrigued by your suggestion for how to estimate and account for
>> the bias in the mean of the EBE distribution.
>>
>> In the usual ETA on EPS model I might write:
>>
>> ; SD of residual error for mixed proportional and additive random
effects
>> PROP=THETA(1)*F
>> ADD=THETA(2)
>> SD=SQRT(PROP*PROP + ADD*ADD)
>> Y=F + EPS(1)*SD*EXP(ETA(1))
>>
>> where EPS(1) is distributed mean zero, variance 1 FIXED
>> and ETA(1) is the between subject random effect for residual error
>>
>> You seem to be suggesting:
>> ETABAR=THETA(3)
>> Y=F + EPS(1)*SD*EXP(ETA(1)) * ETABAR*EXP(ETA(2))
>>
>> It seems to me that the variance of ETA(1) will be confounded with
the
>> variance of ETA(2). Would you please explain more clearly (with an
>> explicit NM-TRAN code fragment if possible) what you are suggesting?
>>
>> Best wishes,
>>
>> Nick
>>
>> Xia Li wrote:
>>> Hi Jakob,
>>> Thank you very much for the information adding an "eta on epsilon".
This
>> is
>>> what I did in my research and I am glad to see people in
Pharmacometrics
>> is
>>> using it.
>>>
>>> And in Bayesian analysis, adding one more stage for ETA, i.e
>>> ETA=ETABAR*exp(eta2), eta2~N(0,omega2) will allow the deviation from
zero
>>> and shrinkage of ETA.
>>>
>>> Again, thanks all for your input.:)
>>>
>>> Best Regards,
>>> Xia
>>>
>>> Xia Li
>>> Mathematical Science Department
>>> University of Cincinnati
>>>
>>
======================================
Xia Li
Mathematical Science Department
University of Cincinnati