Akaike information criterion

From: "James Bailey" <James_Bailey@Emory.org> Subject: Akaike information criterion Date: Thu, 12 Jul 2001 16:58:56 -0500 To all: In selecting an optimal model using the Akaike information criterion should one equate the number of parameters to the sum of the number of structural (clearances, volumes) and error (etas) parameters or should one simply use the number of structural parameters. Jim Bailey

From: "Sale, Mark" <ms93267@GlaxoWellcome.com> Subject: RE: Akaike information criterion Date: Fri, 13 Jul 2001 08:31:00 -0400 Jim, Something I've wondered about as well. My view is that you can alway convert an OMEGA to a THETA, as in $PK S1 = THETA(1) + ETA(1) $THETA (0,1) $OMEGA (0.3) IS THE SAME AS $PK S1 = THETA(1) + THETA(2)*ETA(1) $THETA (0,1) (0,0.3) $OMEGA (1,FIXED) So, why not treat them the same? Mark

From: "Bachman, William" <bachmanw@globomax.com> Subject: RE: Akaike information criterion Date: Fri, 13 Jul 2001 08:38:23 -0400 You count all parameters - fixed and random effect parameters (thetas, etas and epsilons) in calculating AIC. AIC = OFV + 2p, where p is total number of parameters. Bill

From: cng@imap.unc.edu Subject: Re: RE: Akaike information criterion Date: Fri, 13 Jul 2001 10:45:14 -0400 (Eastern Daylight Time) If I understand correctly, the single-sample statistics (for linear model ) like AIC, SBC, MDL, FPE, Mallow's Cp etc. can only be used as crude estimates of generalization error in nonlinear models when you have a "large" training set. Why use AIC? Did anyone try SBC or MDL (Minimum Description Length Principle). Among the simple generalization estimators that do not require the noise variance to be known, SBC often work well (at least in neural network). Shao (1995) showed that in linear model (at least), SBC provides consistent sub-set selection, while AIC dose not. That is, SBC will choose the "best" subset with probability approaching one as the size of the training set goes to infinity. AIC has an asymptotic probability of one of choosing a good subset , but less than one of choosing the best subset (Stone 1979). Many simulation studies have also found that AIC overfits badly in small samples, and that SBC works well. MDL has been showed to be closely related to SBC. Did anyone know a study that compare the model selection criterion (i.e SBC, AICs) in NOMEM model selection? Thanks. Chee Ng

From: "Bachman, William" <bachmanw@globomax.com> Subject: RE: RE: Akaike information criterion Date: Fri, 13 Jul 2001 10:51:55 -0400 See: Comparison of the Akaike Information Criterion, the Schwarz Criterion and the F Test as Guides to Model Selection Sheiner, Beal, & Ludden J. Pharmacokin. Biopharm.,1994,(22),431-445 Bill

From: "Gibiansky, Ekaterina" <gibianskye@globomax.com> Subject: RE: Akaike information criterion Date: Fri, 13 Jul 2001 10:58:21 -0400 I used SBC for model selection in NONMEM, and actually compared it with AIC, not in the simulation studies though, but with actual data. With large data sets AIC tends to choose overestimated models, keeping many more covariates, than SBC. SBC seemed to perform well. Katya Ekaterina Gibiansky, PhD Senior Scientist GloboMax LLC 7250 Parkway Drive, Suite 430 Hanover, MD 21076 Voice (410) 782-2234 FAX (410) 712-0737 E-mail: gibianskye@globomax.com

From: "Gibiansky, Ekaterina" <gibianskye@globomax.com> Subject: RE: Akaike information criterion Date: Mon, 16 Jul 2001 09:06:39 -0400 Sorry, Bill, overparameterized, of course. Katya