Re: LLR test, AIC, BIC

From: Leonid Gibiansky lgibiansky@emmes.com Subject: Re: [NMusers] LLR test, AIC, BIC Date: 10/22/2003 10:06 AM Nick, This randomization method might be the most appropriate for the problem, but even with the most advanced hardware/software combination you will not be able to apply this procedure to each model comparison step of the modeling process (the number of this steps can easily be in dozens for the base model, in hundreds for the covariate model, each step running anywhere from few minutes to few hours). Therefore, there should be a way (you may call it quick and dirty way) to decide whether to accept the model or make it more complicated. Then the question is not whether to use some quick criteria based on the objective function or use randomization test, but rather which of the criteria (LLC, AIC, BIC) to use and how to compute them correctly. Correctly here means "with the highest probability that crude criteria will correctly approximate true distribution". It would interesting to compare crude approach with randomization approach to extract some recommendations on when and how to use crude approach. Leonid