Re: Simulation vs. actual data

From: "Nick Holford" n.holford@auckland.ac.nz Subject: Re: [NMusers] Simulation vs. actual data Date: Sat, June 25, 2005 2:53 am Ken, Thanks you for offering some definitions for various statistical intervals. I offered the following to Toufigh to describe what I have been calling a prediction interval. "There are two ways to define distributions. The most common is to assume a theoretical distribution e.g. normal. If you make this assumption you can then use the estimates of the distribution parameters to predict intervals which include 90% of the distribution e.g. 1.65*StDev. Note that you make a big assumption that the values are normally distributed and in practice this is often not the case e.g. if your clearance is log normally distributed then the distribution of predicted Css will not be normally distributed. I prefer not to make the normal assumption. An alternative is to use an empirical distribution based on data. By simulating 1000 observations at a particular time you have an empirical distribution (emp_dist) of the predicted values. If you find the percentile(emp_dist,0.05) and percentile(emp_dist,0.95) then you define the empirical 90% prediction interval. I use the percentile() function in Excel to do this." It seems that the method I describe comes closest to what you call a tolerance interval i.e. generate an empirical distribution by simulating DVs from the model using the design of the data used to estimate the model parameters. The method might be called a degenerate tolerance interval (following the terminology of Yano, Beal & Sheiner 2001) because it does not account for uncertainty in the parameters. Would you agree? I am not clear about what the distinction is between your tolerance and predictive intervals. It seems that predictive intervals are obtained with "specific designs" perhaps different from the original data. Is this the only difference between a tolerance and a predictive interval ie. the design used to generate the simulations? Confidence intervals seem to be the same as tolerance intervals but use a large number of simulated DVs to get some kind of asymptotic behaviour. You don't say how many replications are used to generate tolerance and predicitive intervals. Am I correct in saying that confidence intervals and tolerance intervals differ only in the number of replications used to generate the empirical distribution? Does the method I describe to Toufigh correspond more to a confidence interval than a tolerance interval because of the large number (1000) replications? Yano Y, Beal SL, Sheiner LB. Evaluating pharmacokinetic/pharmacodynamic models using the posterior predictive check. J Pharmacokinet Pharmacodyn 2001;28(2):171-92 -- Nick Holford, Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand email:n.holford@auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556 http://www.health.auckland.ac.nz/pharmacology/staff/nholford/