RE: OMEGA matrix

Hi All, I agree with everything that Marc and Douglas have pointed out. I too do not advise building the omega structure based on repeated likelihood ratio tests. The approach I take is more akin to what Joe had suggested earlier using SAEM to fit the full block omega structure and then look for patterns in the estimated omega matrix. Even with FOCE estimation I will often fit a full block omega structure just to look for such patterns. The full block omega structure may be over-parameterized and sometimes may not even converge. Nevertheless, as a diagnostic run it can be useful for uncovering patterns that may lead to reduced omega structures with more stable model fits (i.e., not over-parameterized). I’m not necessarily driven to find a parsimonious omega structure as I’ll certainly err on the side of including additional elements in omega provided there is sufficient support to estimate these parameters (i.e., a stable model fit). For example, I will select a full omega structure regardless of the magnitude of the correlations if the model is stable and not over-parameterized. I have no issue with those who want to identify a parsimonious omega structure, however, I still maintain that a diagonal omega structure often is not the most parsimonious. I also agree with Marc’s comment that we must judge parsimony relative to the intended purpose of the model. If we are only interested in our model to predict central tendency, then a diagonal omega structure may be all that is needed. I would contend, however, that we often want to use our models for more than just predicting central tendency. If we perform VPCs, cross-validation, or external validations on independent datasets, but the statistics we summarize to assess predictive performance are only those involving central tendency then we’re not really going to get a robust assessment of the omega structure. To evaluate the omega structure we need to use VPC statistics that describe variation and other percentiles besides the median. My impression is that we aren’t as rigorous in our assessments of whether our models can adequately describe the variation in our data. As I stated earlier, I see so many standard VPC plots where virtually 100% of the observed data are contained well within the 5th and 95th percentiles. The presenter will often claim that these VPC plots support the adequacy of the predictions but clearly the model is over-predicting the variation. The over-prediction of the variation may or may not be related to the omega structure as it could also be related to skewed or non-normal random effect distributions. However, if a diagonal omega structure was used and I saw this over-prediction in the variation in a VPC plot, one of the first things I would do is re-evaluate the omega structure and see if an alternative omega structure can lead to improvements in predicting these percentiles. Best, Ken