RE: covariate selection question

From: "Kowalski, Ken" Ken.Kowalski@pfizer.com Subject: RE: [NMusers] covariate selection question Date: Fri, 20 Jan 2006 13:06:24 -0500 Mark, I certainly did not mean to imply that stepwise procedures can find the most parsimonious model...so I don't think we have any disagreement. Of course we should not equate the most parsimonious model with the true model. Even though we may have sufficient power to detect certain covariate effects, the power to select the true correct model (correct combination of covariates), assuming that it is in within the search space of hierachical models being investigated, is often very poor regardless of the covariate selection procedure (including the WAM). In my paper on the WAM I did a simulation study for a relatively small problem (80 subjects, 400 observations, 9 covariate effects) where the true model was contained within the 2**9=512 possible covariate models. While the power was reasonable to detect indivdidual covariate effects (80-90%) the power to correctly identify the true model out of the 512 possible models was only 11.5%. Of course with larger datasets we may have greater power but if we also substantially increase the search space (number of covariate parameters) this will reduce our power to identify the correct model. Ken