Does anyone have a reference to a publication assessing whether mixed effect modeling (NONMEM in particular) is robust for unbalances studies? I see if in a number of courses (including the original beginners course for NONMEM), but can't find a publication. thanks Mark Sale MD Next Level Solutions, LLC www.NextLevelSolns.com 919-846-9185
unbalanced design
Hi Mark,
This should be true just based on the nature of mixed effects modeling.
If you are not convinced, you may want to try some examples where you
simulate balanced and unbalanced designs and then estimate. :-)
Best Regards,
Susan
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Mark Sale - Next Level Solutions
Sent: Tuesday, September 02, 2008 1:28 PM
To: [email protected]
Subject: [NMusers] unbalanced design
Does anyone have a reference to a publication assessing whether mixed
effect modeling (NONMEM in particular) is robust for unbalances studies?
I see if in a number of courses (including the original beginners course
for NONMEM), but can't find a publication.
thanks
Mark Sale MD
Next Level Solutions, LLC
www.NextLevelSolns.com
919-846-9185
Linear Mixed Models for Longitudinal Data by Geert Verbeke
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744
?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Geert%20Verbek
e , Geert Molenberghs
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744
?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Geert%20Molenb
erghs
Yaning Wang, Ph.D.
Team Leader, Pharmacometrics
Office of Clinical Pharmacology
Office of Translational Science
Center for Drug Evaluation and Research
U.S. Food and Drug Administration
Phone: 301-796-1624
Email: [EMAIL PROTECTED]
"The contents of this message are mine personally and do not necessarily
reflect any position of the Government or the Food and Drug
Administration."
Quoted reply history
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Mark Sale - Next Level Solutions
Sent: Tuesday, September 02, 2008 1:28 PM
To: [email protected]
Subject: [NMusers] unbalanced design
Does anyone have a reference to a publication assessing whether
mixed effect modeling (NONMEM in particular) is robust for unbalances
studies? I see if in a number of courses (including the original
beginners course for NONMEM), but can't find a publication.
thanks
Mark Sale MD
Next Level Solutions, LLC
www.NextLevelSolns.com
919-846-9185
<<left.letterhead>>
Hi,
Its not clear to me what Mark had in mind when he asked if " mixed effect modeling (NONMEM in particular) is robust".
But Susan proposes its just obviously OK <grin> and Yaning suggests reading a book for the simple case of linear models. But what about the real world i.e. non-linear mixed models?
And surely there must be some degree of imbalance that would lead to a non-robust description when using a mixed model? e.g. if one is trying to described a disease progress curve and some people are followed long enough to identify an exponential shape while others are followed for a shorter time and appear to have a linear shape then wouldn't there be some bias in the resulting estimates describing the curve depending on the mix of short or long follow up times?
Nick
Willavize, Susan wrote:
Hi Mark,
This should be true just based on the nature of mixed effects modeling. If you are not convinced, you may want to try some examples where you simulate balanced and unbalanced designs and then estimate. J
Best Regard
Wang, Yaning wrote:
> Linear Mixed Models for Longitudinal Data by Geert Verbeke
>
> http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Geert%20Verbeke,
> Geert Molenberghs
>
> http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Geert%20Molenberghs
>
> Yaning Wang, Ph.D. Team Leader, Pharmacometrics Office of Clinical Pharmacology
>
> Office of Translational Science
> Center for Drug Evaluation and Research
> U.S. Food and Drug Administration
> Phone: 301-796-1624
> Email: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>
> "The contents of this message are mine personally and do not necessarily reflect any position of the Government or the Food and Drug Administration."
>
> ------------------------------------------------------------------------
>
> *From:* [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED] *On Behalf Of *Mark Sale - Next Level Solutions
>
> *Sent:* Tuesday, September 02, 2008 1:28 PM
> *To:* [email protected]
> *Subject:* [NMusers] unbalanced design
>
> Does anyone have a reference to a publication assessing whetheor unbalances studies? I see if in a number of courses (including the original beginners course for NONMEM), but can't find a publication.
>
> thanks
>
> Mark Sale MD
> Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Hi,
In Nick's example, the bias in disease progression parameters may indeed be
higher in the unbalanced design compared to the full, more extensive, design
in all subjects. However, that would in my mind come from data sparseness.
Bias would be expected to be even larger when all subjects have the sparser
design if for example the FOCE method is used. Whenever data per subject
becomes sparser, the FOCE method becomes more like the FO method and
therefore in general more biased in the parameter estimates.
Thus, robustness would decrease in the order "rich balanced design",
"rich+sparse unbalanced design", "sparse unbalanced design". Apart from this
effect I know of no reason to expect unbalanced designs not to be robust if
the model is correctly specified.
Best regards,
Mats
Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003
Quoted reply history
-----Original Message-----
From: owner-nmusers
Behalf Of Nick Holford
Sent: Tuesday, September 02, 2008 10:03 PM
To: Wang, Yaning
Cc: Mark Sale - Next Level Solutions; nmusers
Subject: Re: [NMusers] unbalanced design
Hi,
Its not clear to me what Mark had in mind when he asked if " mixed
effect modeling (NONMEM in particular) is robust".
But Susan proposes its just obviously OK <grin> and Yaning suggests
reading a book for the simple case of linear models. But what about the
real world i.e. non-linear mixed models?
And surely there must be some degree of imbalance that would lead to a
non-robust description when using a mixed model? e.g. if one is trying
to described a disease progress curve and some people are followed long
enough to identify an exponential shape while others are followed for a
shorter time and appear to have a linear shape then wouldn't there be
some bias in the resulting estimates describing the curve depending on
the mix of short or long follow up times?
Nick
Willavize, Susan wrote:
Hi Mark,
This should be true just based on the nature of mixed effects modeling.
If you are not convinced, you may want to try some examples where you
simulate balanced and unbalanced designs and then estimate. J
Best Regard
Wang, Yaning wrote:
>
>
> Linear Mixed Models for Longitudinal Data by Geert Verbeke
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Verbeke,
> Geert Molenberghs
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Molenberghs
>
>
>
> Yaning Wang, Ph.D.
> Team Leader, Pharmacometrics
> Office of Clinical Pharmacology
> Office of Translational Science
> Center for Drug Evaluation and Research
> U.S. Food and Drug Administration
> Phone: 301-796-1624
> Email: yaning.wang
>
> "The contents of this message are mine personally and do not
> necessarily reflect any position of the Government or the Food and
> Drug Administration."
>
>
>
> ------------------------------------------------------------------------
> *From:* owner-nmusers
> [mailto:owner-nmusers
> Level Solutions
> *Sent:* Tuesday, September 02, 2008 1:28 PM
> *To:* nmusers
> *Subject:* [NMusers] unbalanced design
>
>
> Does anyone have a reference to a publication assessing whetheor
> unbalances studies? I see if in a number of courses (including the
> original beginners course for NONMEM), but can't find a publication.
> thanks
>
>
> Mark Sale MD
> Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Hi,
In Nick's example, the bias in disease progression parameters may indeed be
higher in the unbalanced design compared to the full, more extensive, design
in all subjects. However, that would in my mind come from data sparseness.
Bias would be expected to be even larger when all subjects have the sparser
design if for example the FOCE method is used. Whenever data per subject
becomes sparser, the FOCE method becomes more like the FO method and
therefore in general more biased in the parameter estimates.
Thus, robustness would decrease in the order "rich balanced design",
"rich+sparse unbalanced design", "sparse unbalanced design". Apart from this
effect I know of no reason to expect unbalanced designs not to be robust if
the model is correctly specified.
Best regards,
Mats
Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Nick Holford
Sent: Tuesday, September 02, 2008 10:03 PM
To: Wang, Yaning
Cc: Mark Sale - Next Level Solutions; [email protected]
Subject: Re: [NMusers] unbalanced design
Hi,
Its not clear to me what Mark had in mind when he asked if " mixed
effect modeling (NONMEM in particular) is robust".
But Susan proposes its just obviously OK <grin> and Yaning suggests
reading a book for the simple case of linear models. But what about the
real world i.e. non-linear mixed models?
And surely there must be some degree of imbalance that would lead to a
non-robust description when using a mixed model? e.g. if one is trying
to described a disease progress curve and some people are followed long
enough to identify an exponential shape while others are followed for a
shorter time and appear to have a linear shape then wouldn't there be
some bias in the resulting estimates describing the curve depending on
the mix of short or long follow up times?
Nick
Willavize, Susan wrote:
Hi Mark,
This should be true just based on the nature of mixed effects modeling.
If you are not convinced, you may want to try some examples where you
simulate balanced and unbalanced designs and then estimate. J
Best Regard
Wang, Yaning wrote:
>
>
> Linear Mixed Models for Longitudinal Data by Geert Verbeke
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Verbeke,
> Geert Molenberghs
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Molenberghs
>
>
>
> Yaning Wang, Ph.D.
> Team Leader, Pharmacometrics
> Office of Clinical Pharmacology
> Office of Translational Science
> Center for Drug Evaluation and Research
> U.S. Food and Drug Administration
> Phone: 301-796-1624
> Email: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>
> "The contents of this message are mine personally and do not
> necessarily reflect any position of the Government or the Food and
> Drug Administration."
>
>
>
> ------------------------------------------------------------------------
> *From:* [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] *On Behalf Of *Mark Sale - Next
> Level Solutions
> *Sent:* Tuesday, September 02, 2008 1:28 PM
> *To:* [email protected]
> *Subject:* [NMusers] unbalanced design
>
>
> Does anyone have a reference to a publication assessing whetheor
> unbalances studies? I see if in a number of courses (including the
> original beginners course for NONMEM), but can't find a publication.
> thanks
>
>
> Mark Sale MD
> Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Hi -
Here "robust" seems to be being used to have the meaning "unbiased"
(or perhaps asymptotically unbiased or even consistent).
The more usual statistical meaning
of robust with respect to an estimation method is that the method
is relatively resilient to small departures from model assumptions,
such as some degree of non-normality of residuals or random effects. For
example,
the mean is not a robust measure of central tendency of a distribution,
whereas the median is robust. Most classical maximum likelihood-based
estimation methods based on normality assumptions are not robust,
and in this sense none of the usual NONMEM parametric methods is robust,
regardless of the experimental design.
Non-parametric methods (e.g. the median is a nonparametric estimator)
tend to be more robust.
In the sense of being asymptotially unbiased or the stronger condition
of being consistent, NONMEM FOCE and Laplacian methods
are (weakly) consistent in the sense that they will converge to
the true parameter values as (loosely speaking, since
there are degenerate cases where this is not true)
both the number of subjects and the amount of data per subject
increase without bound. The are not strongly consistent in the sense
that biased estimates will still be produced if the amount
of data increases without bound but either the number
of subjects or amount of data per subject remains bounded.
The FO method is biased regardless of the amount of data.
In fact, FO results often become worse as the amount of data per subject
increases . Alan Schumitzky has a nice example of this
in which he obtains a lower bound on the FO bias for a particular
model where this bound in fact increases with the amount of data per subject.
The problem is that the joint likelihood function for each individual
becomes more and more peaked around its mode (the empirical Bayes estimate),
but the FO method is based on an implicit quadratic
extrapolation to estimate the mode position, and the quality of this
extrapolation becomes poorer as the joint likelihood becomes more peaked.
Robert H. Leary, PhD
Principal Software Engineer
Pharsight Corp.
5520 Dillard Dr., Suite 210
Cary, NC 27511
Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871
> This email message (including any attachments) is for the sole use of the
> intended recipient and may contain confidential and proprietary information.
> Any disclosure or distribution to third parties that is not specifically
> authorized by the sender is prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies of
> the original message.
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Mats Karlsson
Sent: Tuesday, September 02, 2008 22:53 PM
To: 'Nick Holford'; 'Wang, Yaning'
Cc: 'Mark Sale - Next Level Solutions'; [email protected]
Subject: RE: [NMusers] unbalanced design
Hi,
In Nick's example, the bias in disease progression parameters may indeed be
higher in the unbalanced design compared to the full, more extensive, design
in all subjects. However, that would in my mind come from data sparseness.
Bias would be expected to be even larger when all subjects have the sparser
design if for example the FOCE method is used. Whenever data per subject
becomes sparser, the FOCE method becomes more like the FO method and
therefore in general more biased in the parameter estimates.
Thus, robustness would decrease in the order "rich balanced design",
"rich+sparse unbalanced design", "sparse unbalanced design". Apart from this
effect I know of no reason to expect unbalanced designs not to be robust if
the model is correctly specified.
Best regards,
Mats
Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Nick Holford
Sent: Tuesday, September 02, 2008 10:03 PM
To: Wang, Yaning
Cc: Mark Sale - Next Level Solutions; [email protected]
Subject: Re: [NMusers] unbalanced design
Hi,
Its not clear to me what Mark had in mind when he asked if " mixed
effect modeling (NONMEM in particular) is robust".
But Susan proposes its just obviously OK <grin> and Yaning suggests
reading a book for the simple case of linear models. But what about the
real world i.e. non-linear mixed models?
And surely there must be some degree of imbalance that would lead to a
non-robust description when using a mixed model? e.g. if one is trying
to described a disease progress curve and some people are followed long
enough to identify an exponential shape while others are followed for a
shorter time and appear to have a linear shape then wouldn't there be
some bias in the resulting estimates describing the curve depending on
the mix of short or long follow up times?
Nick
Willavize, Susan wrote:
Hi Mark,
This should be true just based on the nature of mixed effects modeling.
If you are not convinced, you may want to try some examples where you
simulate balanced and unbalanced designs and then estimate. J
Best Regard
Wang, Yaning wrote:
>
>
> Linear Mixed Models for Longitudinal Data by Geert Verbeke
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Verbeke,
> Geert Molenberghs
>
http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?%5F
encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Molenberghs
>
>
>
> Yaning Wang, Ph.D.
> Team Leader, Pharmacometrics
> Office of Clinical Pharmacology
> Office of Translational Science
> Center for Drug Evaluation and Research
> U.S. Food and Drug Administration
> Phone: 301-796-1624
> Email: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>
> "The contents of this message are mine personally and do not
> necessarily reflect any position of the Government or the Food and
> Drug Administration."
>
>
>
> ------------------------------------------------------------------------
> *From:* [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] *On Behalf Of *Mark Sale - Next
> Level Solutions
> *Sent:* Tuesday, September 02, 2008 1:28 PM
> *To:* [email protected]
> *Subject:* [NMusers] unbalanced design
>
>
> Does anyone have a reference to a publication assessing whetheor
> unbalances studies? I see if in a number of courses (including the
> original beginners course for NONMEM), but can't find a publication.
> thanks
>
>
> Mark Sale MD
> Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
>
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)923-6730 fax:+64(9)373-7090
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford