NONMEM Estimation algorithms

From: Bonate, Peter pbonate@ilexonc.com Subject: [NMusers] NONMEM Estimation algorithms Date: Thu, May 20, 2004 8:59 am Dear all, I am having some trouble understanding some of the estimation algorithms used in NONMEM and was hoping someone would take the time to help me out. Here are some of my questions: 1.) How does what NONMEM does with FOCE differ from the Lindstrom and Bates conditional algorithm (1990)? 2.) Is the Laplacian method in NONMEM the same as the Laplace method suggested by Wolfinger (1993)? Some have indicated that the Laplacian method in NONMEM is a second-order Taylor series approximation about the nonlinear model itself, like FO-approximation (Racine-Poon and Wakefield, 1998). But it looks from the NONMEM manuals that the -2LL for the Laplace option is a second-order Taylor series approximation about the likelihood function itself. Which is it? And this leads to my third question. 3.) Where does the equation on page 5 in the Part VII manuals come from? How is it derived? I have seen no other place in any paper where this form of the -2LL is presented. So on a related note are conditional algorithms published anywhere outside of the NONMEM manuals. It seems that the FO algorithm is described in detail in many papers, but the algorithms that NONMEM actually uses are not published anywhere. It this correct? Thanks for your time if you can help me. Pete bonate Peter L. Bonate, PhD, FCP Director, Pharmacokinetics ILEX Oncology 4545 Horizon Hill Blvd San Antonio, TX 78229 phone: 210-949-8662 fax: 210-949-8219 email: pbonate@ilexonc.com

From: Wang, Yaning WangYA@cder.fda.gov Subject: RE: [NMusers] NONMEM Estimation algorithms Date: Fri, May 21, 2004 8:39 am Hi, Peter: Please see the following comments (they are just my personal opinions). 1) On page 174 of Marie Davidian and David Giltina's book, Nonlinear Models for Repeated Measurement data (Chapman & Hall, 1995), NONMEM's FOCE was described as "a full normal theory maximum likelihood version of the method of Lindstrom and Bates". I don't quite agree with this statement. In my opinion, L & B's method can be described as a first order Taylor approximation of the nonlinear model around current estimates of both theta and eta (I will use NONMEM terminology for fixed-effect parameters and random effect parameters). NONMEM's FOCE without interaction can be described as a first order Taylor approximation of the nonlinear model around the posterior mode of eta. FOCE without interaction can also be derived from Laplacian approximation. But this is not the case for FOCE with interaction (see chapter 7 in Edward Vonesh and Vernon Chinchilli's book, Linear and Nonlinear Models for the Analysis of Repeated Measures, Marcel Dekker 1997). Similarly, NONMEM's FO method (expansion of nonlinear model around 0 for eta) can also be derived from Laplacian approximation. I believe that NONMEM's FOCE is superior to L&B's method in terms of approximation accuracy, but L&B's method can provide restricted maximum likelihood estimators. In Pinhero and Bates' book (chapter 7), Mixed-Effect Models in S and S-plus, what they called "modifed Laplacian approximation" is basically FOCE in NONMEM in my opinion. The accuracy rank of different approximation methods should be Adaptive Gaussian Approximation (SAS PROC NLMIXED)>Laplacian Approximation (NONMEM Laplacian)>FOCE (NONMEM)>L&B's Method (Splus nlme)>FO(NONMEM and SAS). The superiority of Laplacian in NONME over FOCE is certain for "without interaction" case. I am not sure about "with interaction" case since only FOCE has this option while Laplacian doesn't (at least for the current version of NONMEM). Despite its better approximation accuracy, Laplacian method will force "no interaction". The impact of this on parameter estimation is unknown. In fact, most of time our residual models, such as proportional model or combination model, indicate interaction. 2) No, I don't think the two Laplacian methods are the same even though the reinterpretation of Wolfinger's method by Marie Davidian and David Giltina (chapter 6.3.3) made them look so similar. First, Wolfinger's method treated the marginal likelihood as an integral with respect to both the random effects and the fixed effects with a tacitly present flat prior for the fixed effects while the Laplacian method in NONMEM treated the marginal likelihood as an integral with respect to the random effects only. Second, Wolfinger's method led to the restricted maximum likelihood version of L&B's LME step while the Laplacian method in NONMEM (also described in details in Pinhero and Bates' book) cannot lead to restricted maximum likelihood estimators unless another Taylor expansion of the model function was done around the current estimates of theta to make the fixed effects enter the model linearly. In other words, L&B's method involves further approximation compared to FOCE in NONMEM but with the advantage of being able to obtain restricted maximum likelihood estimators. I think there is some misunderstanding about the second-order Taylor expansion in Laplacian method. It is often compared to the first-order expansion to show its better accuracy in approximation. In fact, this second-order Taylor expansion is referring to the Taylor expansion of the log of the integrand that is used to calculate the marginal likelihood (see my derivation of NONMEM VII likelihood for details). If derived from Laplacian approximation, Laplacian, FOCE and FO all use second-order Taylor expansion in the Laplacian approximation step. When there is no interaction, FOCE can be thought of as a first-order Taylor expansion of the nonlinear model around eta^ just like FO is a first-order Taylor expansion of the nonlinear model around 0 for eta. But I have never seen any paper that proved the second-order Taylor expansion in Laplacian method is equivalent to second-order Taylor series approximation of the nonlinear model around eta^. 3) Please see the nonmem.pdf on the following link for detailed derivation of the equation on page 5 of NONMEM VII. I don't know how to attach file to this mail list. So I put it on my webpage. http://www.geocities.com/wangyaning2004/nonmem.pdf I hope this helps. Yaning Wang, PhD Pharmacometrician OCPB/FDA _______________________________________________________