Re: Confidence intervals of PsN bootstrap output

Nick, Jakob, nmusers: At the risk of re-opening an old discussion, I'd like to expand on Nick's important point. From our experience the experience of others, including parameters from all bootstrap runs that report parameter estimates, regardless of minimization of $COV status, is a reasonable thing to do; provided that the modeler conducts a careful and thoughtful evaluation of the model and bootstrap results. One goal should be to understand why minimization, $COV, or boundary problems occur. For example, it is useful to create histograms and a scatterplot matrix of the resulting bootstrap parameter distributions, flagging those runs with minimization problems. Most of the time, these problematic runs represent the tails of the distribution for a less precisely defined parameter. Sometimes the boundary message is just a reflection of the tail of a bootstrap parameter distribution approaching the null value for that parameter. All of these runs are informative about the estimation precision and should be included. This is not always the case, however. In fact, even successful minimization and $COV do not guarantee that the resulting parameter estimates should be included in bootstrap confidence intervals. For example, bootstrapping is often a useful way to identify models with more than 1 unique solution (e.g. flip-flop in PK rate constants). In addition, some runs may represent a local minimum. Another problem can arise if the bootstrap resampling is not stratified on the major covariate(s) or design features of the original data set (e.g. pooled analyses of adult studies and pediatric studies, or pooled analyses of sparsely sampled and extensively sampled studies). In all of these cases, the histograms and scatterplot matrices may reveal a multimodal "posterior" bootstrap distribution for some of the parameters, which should be addressed before using bootstrap results for inferential or simulation purposes. Obviously, it's important for the modeler to think about the problem and to understand what the results mean. Blindly accepting or rejecting runs based on a software warning message, or on another scientist's R script, is not advisable. Best regards, Marc Marc R. Gastonguay, Ph.D. ([email protected]) President & CEO, Metrum Research Group LLC (metrumrg.com) Scientific Director, Metrum Institute (metruminstitute.org)