RE: Describing variability

From:"Kowalski, Ken" Subject: RE: [NMusers] Describing variability Date:Tue, 1 Apr 2003 11:04:04 -0500 All, A successful $COV step is not a bonus. $COV step failures and convergence problems (i.e., rounding errors) are indicative of some form of ill-conditioning or over-parameterization of the model. Granted, such over- parameterized models may indeed fit the data well and sometimes they may result in reasonable estimates but that is not guaranteed. Moreover, such estimates are probably not unique...change the starting values by 10% and you'll probably end up with a different set of estimates that fit the data equally well. One should be extra cautious in interpreting the parameter estimates and using the model for extrapolation when such instability arises. Use of such over-parameterized models for inference should probably be supported by simulation studies on a case-by-case basis. The frequentist-based methods in NONMEM V rely solely on the data in hand to estimate the parameters in the model. If one is fitting a complex model that is not completely supported by the data in hand we have two basic choices: 1) Reduce the complexity of the model to remove ill-conditioning while still providing a good fit to the data, or 2) Make use of additional information regarding the complex model based on prior data and/or beliefs. The second option is basically to take a Bayesian approach. If one has a lot of confidence that the complex model is the correct one and the data is consistent with this model but not rich enough to estimate all the parameters (that's what the rounding errors and $COV step failures are indicating) then one should explicitly make use of the confidence in this information. If one has a lot of confidence in the value of one or more parameters that are not well estimated with the existing data then consider fixing it to that value to remove the ill-conditioning. This can be done more formally taking into account uncertainty in one's prior beliefs by using a Bayesian approach. Thinking of successful convergence and $COV steps as a luxury (i.e., nice to have but not necessary) is not a good practice. If you tend to build complex models that exceed the information content of the data but you KNOW your model is right based on the science, then use a more appropriate tool that incorporates this knowledge. To fit the complex model using a frequentist-based method without incorporating your prior knowledge and 'pretending' that the data can accurately and precisely estimate all of the parameters is risky. JMHO Ken