Question: Number of ETA and EPS in statistic significance

From: takeshi.tajima@pharma.Novartis.com Date: Wed, 26 May 1999 13:18:22 +0900 Subject: Question: Number of ETA and EPS in statistic significance Dear NONMEM users, Please answer to my question below. ----------------------------------------------------------------- We can statistically evaluate the change of values of object function (OBJ) when the number of THETA is altered. Example, in addition of another THETA, change of more than 3.84 in OBJ is significant. How about the number of ETA and EPS ? Can we peform statistical evaluation using the change of OBJ with an increase or a decrease of the number of ETA and EPS ? ----------------------------------------------------------------- Best regards, Takeshi Tajima Novartis Pharma K.K. Tsukuba Research Institute

Date: Wed, 26 May 1999 08:42:05 +0200 From: Mats Karlsson <Mats.Karlsson@biof.uu.se> Subject: Re: Question: Number of ETA and EPS in statistic significance Dear Takeshi, As I understand it, the same test can be performed for ETA and EPS. However, there are two things to keep in mind. 1) The test is approximate for THETA and even more so for variance components. As the information in data increase, the test statistics for variance components are converging towards the true slower than for THETA. 2) Even for THETA's, the test can behave badly even under relatively normal situations. Using simulations, we have compared nominal to true p-levels and found that the true p-level for 3.84 sometimes can be >0.4, and that several foctors influence the difference between the nominal and true p-levels. A presentation on this topic will be given by Ulrika Wahlby at the upcoming PAGE meeting (Saintes, France June 17-18). For ETA's and EPS's we have not yet generated the same information. Best regards, Mats Karlsson ________________________________________________________ Mats Karlsson, PhD Uppsala University Div. of Biopharmaceutics and Pharmacokinetics Dept. of Pharmacy/ Box 580, SE-751 23 Uppsala, Sweden Internet: mats.karlsson@biof.uu.se Phone: +46 18 471 41 05 Fax: +46 18 471 40 03

From: "Piotrovskij, Vladimir [JanBe]" <VPIOTROV@janbe.jnj.com> Subject: RE: Question: Number of ETA and EPS in statistic significance Date: Thu, 27 May 1999 09:16:47 +0200 Dear Takeshi, As the matter of fact, we can only very roughly evaluate statistical significance of changes in minimum objective function value (MOF). If we fit two nested models, and one of them is a reduced version of the other, the difference in MOF being the difference in -2*max(LogLik) is asymthotically Chi-squared distributed with q degrees of freedom where q is the number of skipped parameters (normally one). Only with the number of observations tend to infinity the corresponding P-values are correct, and the difference of 3.84 can be considered as significant (P<0.05). In any real situation it is always preferable to apply more conservative tests: 6.6 (P<0.01) or even 7.9 (P<0.005). The situation with random effect parameters, as mentioned by Mats, is much more complicated. In any case, the inclusion of random effects for structural parameters in the model should reduce the residual (unexplained) variability in the dependent variable, and I would suggest this as one of the most important model selection criterion. Best regards, Vladimir Vladimir Piotrovsky, PhD Clinical Pharmacokinetics, ext 5463 Janssen Research Foundation 2340 Beerse, Belgium e-mail: vpiotrov@janbe.jnj.com

Date: Thu, 27 May 1999 16:46:17 -0500 (EDT) From: Ferrin Harrison 301-827-3118 FAX 301-443-9279 <HARRISONF@cder.fda.gov> Subject: Map from nominal to estimated p-values I'd appreciate references and hope Mats will continue research in this area of NONMEM. I'm familiar with the work of Cox and others on the importance of comparing nested models, and of having a fittable supermodel which is a superset of all considered models when feasible, and agree with the comments on that. I'm a statistician and expect to disagree with a fair number of modelers on the importance of and proper approach to developing parsimonious models; and to vary my own approach according to the data and purpose of analysis. The rule of thumb I was given was that a nominal p=.05 might be a "real" p=.10. A more detailed map, and some exploration of whether within and between patient variabilities cost the same or different amounts from location parameters would be appreciated. For example, for single trial submission in place of two trials, I might want to know what nominal p-value maps to a "real" p=.00125. I suppose the cost might depend on the quantity of patients as well for between patient variabilities. One way to formulate the problem is, what price in -2*max(pseudo LogLik) must be paid to add one parameter to the model at a desired significance level? Please continue!