RE: order of covariate inclusion -> avoiding stepwise approaches -> abandoning exploratory analysis?

From: mark.e.sale@gsk.com Subject: RE: [NMusers] order of covariate inclusion -> avoiding stepwise approaches -> abandoning exploratory analysis? Date: 9/29/2003 8:18 AM Nice try at diplomacy Marc, but I'm going to be refractory. We have preliminary data, and Rob Bies has presented some data that show that forward addition/backward elimination rarely (if ever) gives the best answer. Again, my interest is in finding the best answer. The assumption behind any step-wise method is that the search space is monotonically down hill, that there are no interactions between the different effects being considered. That is probably never true. The search spaces we examine are very complicated, not only among covariates, but between structural effects, random effects and covariates. I think that the reason we all use forward addition/backward elimination is related to the metaphor, "If the only tool you have is a hammer, then everything looks like a nail" Lets list a few of the nail-like properties of this problem: 1. If you don't think about it too hard, this algorithm can be used for hypothesis testing. Think about it too hard would include all those pesky statistical assumptions, none of hich are true. 2. If you don't think about it too hard, it can be put in a Bayesian framework. 3. There is no reason to believe that it ever gives the right answer, and we have data to support this. Fortunately, other disciplines have thought about efficient ways to search discrete spaces, and the assumptions required for the different methods. we probably should learn from them. For those interested in learning about modern methods of model selection, we have openings in US (North Carolina and Phil), UK and Italy. Mark Sale M.D. Global Director, Research Modeling and Simulation GlaxoSmithKline 5 Moore Drive RTP NC 27709 919-483-1808