Re: Simulation vs. actual data

From: "Nick Holford" n.holford@auckland.ac.nz Subject: Re: [NMusers] Simulation vs. actual data Date: Thu, July 14, 2005 10:09 pm Ken, Thanks for your comprehensive and comprehensible response. I distinguished 2 levels of random effects (PPV and RUV) because there are two kinds of interval that are of interest depending on what random effects are used: a) PPV only -- the 'true' variability in response independent of RUV b) PPV and RUV -- the variability in observations of the response Type a) is what would want to see for a prediction of the true response while type b) is what you want to see when comparing the preidiction with an observation. I can understand almost everything you say except: 1. What does M mean with respect to future observations? Your example suggests that M is the number of subjects rather than the number of observations. But perhaps you mean M is the number of observations conditioned on time and design within a single trial? 2. Why do you use 'degenerative' instead of 'degenerate'? I agree that the non degenerate tolerance interval is preferable over the degenerate case but the dilemma comes when NONMEM spins the dice and won't reveal the results of $COV and the runtimes preclude a non-parametric bootstrap (a common occurrence for me). In this case the degenerate tolerance interval (DTI) seems to be helpful. If approx 90% of observations lie within the 90% DTI then one has a computationally easy way to confirm model performance as a description of the data. In my rather limited experience of visual predictive checks it is common to see more than 90% of observations lying within the 90% DTI which points to a model failure to capture the random effects properly. I have yet to see an example of more than 10% of the data lying outside the 90% DTI which is what you would expect if uncertainty is high. Nick