Re: General question on modeling
Hello Mark & nmusers,
I'm just catching up with the flurry of emails on this topic...
I don't think that anyone mentioned full model approaches to covariate modeling, although we have discussed this topic in detail in past nmusers threads. Frank Harrell's Regression Modeling Strategies text (I'll second Tony's recommendation) advocates this method as an alternative to stepwise methods when the purpose is to estimate the effect of covariates. The text also includes a useful discussion of the choice of modeling strategy as it relates to modeling objectives (e.g. prediction, effect estimation, hypothesis testing).
In our group, we routinely apply full model methods for population PK covariate modeling and have managed to make useful inferences about covariate effects while avoiding stepwise methods and p-values altogether. Some examples of this method will be presented at ASCPT later this week.
I also agree with the sentiment expressed by several contributors that we shouldn't be so concerned with finding the one perfect model. Instead, we should probably spend more time evaluating the impact of model deficiencies on the intended model-based applications and inferences.
In addition to Harrell's book, some relevant references are listed below.
Best regards,
Marc
Marc R. Gastonguay, Ph.D.
Scientific Director, Metrum Institute [www.metruminstitute.org]
President & CEO, Metrum Research Group LLC [www.metrumrg.com]
Email: [EMAIL PROTECTED] Direct: +1.860.670.0744 Main: +1.860.735.7043
1. Ulrika Wählby, E. Niclas Jonsson and Mats O. Karlsson AAPS PharmSci 2002; 4 (4) article 27 ( http://www.aapspharmsci.org ). Comparison of Stepwise Covariate Model Building Strategies in Population Pharmacokinetic-Pharmacodynamic Analysis ****(full model approach is described in Discussion section).
2. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. J Clin Epidemiol. October 1999;52(10):935-942.
3. Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387.
4. Steyerberg EW, Eijkemans MJ, Harrell FE, Jr., Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med. 2000;19(8):1059-1079.
5. M.R. Gastonguay. A Full Model Estimation Approach for Covariate Effects: Inference Based on Clinical Importance and Estimation Precision. The AAPS Journal; 6(S1), Abstract W4354, 2004. ( http:// metrumrg.com/publications/full_model.pdf)
6. Balaji Agoram; Anne C. Heatherington; Marc R. Gastonguay. Development and Evaluation of a Population Pharmacokinetic- Pharmacodynamic Model of Darbepoetin Alfa in Patients with Nonmyeloid Malignancies Undergoing Multicycle Chemotherapy. AAPS PharmSci Vol.: 8, No.: 3, 2006
Quoted reply history
On Mar 19, 2007, at 3:34 PM, AJ Rossini wrote:
> I'd highly recommend reading Frank Harrell's book on Regression Modeling if you think that stepwise regression makes any sense. While much of the book applies to linear and generalized linear (i.e. categorical, etc) regression models, nonlinear models (and mixed effects models) would generally fall into the "well, if the simple case was like that, it can't be any simpler for the harder cases..."... Frank demonstrates some of the reasons that p- values from models generated using stepwise modeling are fairly useless (i.e. don't
>
> follow the behavior you'd expect from p-values).
>
> The literature to start looking at would be modern variable selection
>
> techniques for linear regression, i.e. work at Stanford Statistics by Hastie, Tibshirani, and their collaborators and former grad students (LASSO, LARS,
>
> elastic nets, and similar approaches).
>
> On Monday 19 March 2007 19:32, Mark Sale - Next Level Solutions wrote:
>
> > Dear Colleagues,
> >
> > I've lately been reviewing the literature on model building/ selection
> >
> > algorithms. I have been unable to find any even remotely rigorous
> > discussion of the way we all build NONMEM models. The structural
> > first, then variances/forward addition/backward elimination is
> > generally mentioned in a number of places (Ene Ettes in Ann
> >
> > Pharmacother, 2004, Jaap Mandemas series on POP PK series J PK Biopharm
> >
> > in 1992, Jose Pinheiros paper from the Joint Stats meeting in 1994,
> > Peter Bonates AAPS journal article in 2005, Mats Karlsons AAPS
> > PharmSci, 2002, the FDA guidance on Pop PK). It is most explicitly
> > stated in the NONMEM manuals (Vol 5, figure 11.1) - without any
> > reference. From the NONMEM manuals it is reproduced in many courses,
> > and has become axiomatic. I've looked at the stats literature on
> > forward addition/backwards elimination in both linear and logistic
> > regression, where it is at least formally discussed (with some
> >
> > disagreement about whether it is "correct"). But, I am unable to find
> >
> > any justification for the structural first, then covariates (drive by
> > post-hoc plots), then variance effects approach we use (I'm sure many
> > people will point out that it is not nearly that linear a process,
> >
> > although in figure 11.1, Vol 5 of the NONMEM manuals, it is depicted as a step-by-step algorithm, without any looping back). Can anyone point
> >
> > me to any rigorous discussion of this model building strategy?
> >
> > Mark Sale MD
> > Next Level Solutions, LLC
> > www.NextLevelSolns.com
>
> --
> best,
> -tony
>
> [EMAIL PROTECTED]
> Muttenz, Switzerland.
>
> "Commit early,commit often, and commit in a repository from which we can
>
> easily
> roll-back your mistakes" (AJR, 4Jan05).