RE: $OMEGA blocks and log-likelihood profiling

From: "Mats Karlsson" mats.karlsson@farmbio.uu.se> Subject: RE: [NMusers] $OMEGA blocks and log-likelihood profiling Date: Sat, June 5, 2004 3:27 am Dear Ken, Jeff, Nick and all, It seems that in the latest discussion, Nick's original observation that parameter estimates were the same, regardless of successful COV step or not has been forgotten. Thus, Ken's suggestion below, that somehow the 7% data sets with successful COV step may contain some information that the other 93% data sets would not, does not seems plausible. If that had been the case, at least one parameter would have displayed a different mean and/or considerably more variability in the runs with failed COV-step than in those with successful COV step. Jeff is worried by so many bootstrap runs fail to converge and that therefore the bootstrap distribution of parameter estimates would be censored. However, no censoring appears to have taken place as Nick does report on the distribution of *all* parameter estimates (even non successful terminations such as "Rounding Errors dominating" will provide a set of parameter estimates). As the type of termination (whether successful COV step or not, successful minimization or not) was not of importance for the resulting distribution of parameter estimates, and local minima related to the use of the same initial estimates was not a problem (which I think Nick said it wasn't), I think it much supports Nicks notion that there is no correlation (in this case) between termination status and parameter estimates. On a general note, we know that with poor models (overparametrisation, misspecification) we can see failed convergence or failed covariance steps, whereas for appropriate models, supported by adequate data the opposite is true. We use the convergence and covariance steps of NONMEM to diagnose when we're moving from one type of problem (the appropriate) to another. However, Nick's results strongly suggest that these diagnostics are not appropriate in this case. This should not come as a surprise. We have here a situation where the COV step diagnostic is not perfect. Most of us have had similar experiences before. Nick seems to draw the conclusion that it therefore is useless. I don't agree with this, but still see COV step as a diagnostic with value even if we should be cautious in accepting both positive and negative results from it. Thus, with access to results from a properly carried out bootstrap, I would not care about COV-step failure or not, just as I generally would not pay attention to NONMEM's approximate SEs if I had bootstrap confidence intervals. COV-step is a cheap (in a positive sense) and often informative diagnostic, but yet not as informative as the more expensive bootstrap. Best regards, Mats -- Mats Karlsson, PhD Professor of Pharmacometrics Div. of Pharmacokinetics and Drug Therapy Dept. of Pharmaceutical Biosciences Faculty of Pharmacy Uppsala University Box 591 SE-751 24 Uppsala Sweden phone +46 18 471 4105 fax +46 18 471 4003 mats.karlsson@farmbio.uu.se