Model Building Algorithm

Date: Mon, 24 Apr 2000 15:38:05 -0400 (EDT) From: "Ferrin Harrison 301-827-3213 FAX 301-480-2825" <HARRISONF@cder.fda.gov> Subject: Model Building Algorithm Some good messages have been posted on this thread. Starting from the maximal model and working down, or simply running every conceivable model are allowed. In the linear model with fixed effects format, forward selection and backward elimination should generally give the best model without recourse to running all possible regressions. The nonlinear random effects format is more complex. I would suggest trying the maximal (all possible parameters included) model before going to other methods of model selection, if only to see whether you have enough data for the model to run. An aspect encountered was a kind of interpolation problem which I would expect to be more frequent in the nonlinear random effects format, i.e. forward selection and backward elimination can give different answers, even when nesting of the models is preserved. If the maximal model doesn't run, then adding theta-eta pairs to forward select before reducing would be another interpretation of not working on the etas till the thetas are done; every theta gets an eta till all the thetas are chosen. The CDER Biometric scientific workstations haven't had a speed upgrade since Peck's last big purchase in 1993, so accepting the 10-20 fold increase in CPU time to use FOCE for model selection would be a problem for me. It would take too many months of CPU time to run through the models when reasonably complex, which would be a problem under the PDUFA/FDAMA review time clocks. If you have the CPU time, I'd certainly favor modeling from FOCE from the start, rather than switching to FOCE for a little validation or precise estimation at the end. So while a statistician may be disturbed and apprehensive to find you working on a more difficult model selection problem than linear fixed effects, this statistician certainly would not call it blasphemous.