RE: Confidence intervals of PsN bootstrap output

From: Stephen Duffull Date: July 11, 2011 technical Source: mail-archive.com
Leonid > of them. If some realizations are so special that the model behaves in > an unusual way (with any definition of unusual: non-convergence, not > convergence of the covariance step, parameter estimates at the boundary, > etc.) we either need to accept those as is, or work with each of those > special data sets one by one to push to the parameter estimates that we > can accept, or change the bootstrap procedure (add stratification by > covariates, by dose level, by route of administration, etc.) so that all > data sets behave similarly. I agree with your comments above and there are times that this might be possible. But in essence I hear that BS may become overtly time consuming in some cases if individual BS'd data sets require almost a full modelling procedure to gain appropriate convergence outcomes. This might make a long BS into an almost impossible task. The risk a user then runs is once you know there are runs that don't work as intended you cannot ignore them. If you can't afford the time resource (say a single run takes 12-24 hours) to recover each of these (say 20% of 1000 BS runs) and since you cannot un-know the results you're stuck. Would not a pragmatic view be appropriate? Report the outcomes of the BS runs as they occurred but then select those runs that terminated appropriately? Otherwise BS is too risky... Steve PS All models are wrong :-) -- > >
Quoted reply history
> On 7/10/2011 2:57 PM, Stephen Duffull wrote: > > Nick, Jakob, Marc et al > > > >> Thanks for your helpful comments. I agree with you that any results that > >> are at a boundary should be discarded from the bootstrap distribution. > > > > On the whole I the sentiments in this thread align with anecdotal findings > from my experience. But, I was just wondering how you define your boundaries > for variance and covariance parameters (e.g. OMEGA terms)? > > > > For variance terms, lower boundaries seems reasonably straightforward (e.g. > 1E-5 seems close to zero). Upper boundaries are of course open, for the > variance of a log-normal ETA would 1E+5 or 1E+4 be large enough to be > considered close to a boundary? At what value would you discard the result? > At what correlation value would you discard a result (>0.99,> 0.97...) as > being close to 1. Clearly if this was for regulatory work you could define > these a priori after having chosen any arbitrary cut-off. But the devil here > lies with the non-regulatory work where you may not have defined these > boundaries a priori. > > > > Steve > > -- > > Professor Stephen Duffull > > Chair of Clinical Pharmacy > > School of Pharmacy > > University of Otago > > PO Box 56 Dunedin > > New Zealand > > E: [email protected] > > P: +64 3 479 5044 > > F: +64 3 479 7034 > > W: http://pharmacy.otago.ac.nz/profiles/stephenduffull > > > > Design software: www.winpopt.com > > > >
Jul 05, 2011 Norman Z Confidence intervals of PsN bootstrap output
Jul 05, 2011 Jakob Ribbing Re: Confidence intervals of PsN bootstrap output
Jul 06, 2011 Norman Z Re: Confidence intervals of PsN bootstrap output
Jul 06, 2011 Justin Wilkins Re: Confidence intervals of PsN bootstrap output
Jul 08, 2011 Jakob Ribbing RE: Confidence intervals of PsN bootstrap output
Jul 09, 2011 Jakob Ribbing RE: Confidence intervals of PsN bootstrap output
Jul 09, 2011 Nick Holford Re: Confidence intervals of PsN bootstrap output
Jul 09, 2011 Marc Gastonguay Re: Confidence intervals of PsN bootstrap output
Jul 10, 2011 Stephen Duffull RE: Confidence intervals of PsN bootstrap output
Jul 10, 2011 Leonid Gibiansky Re: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Nick Holford Re: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Justin Wilkins Re: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Mats Karlsson RE: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Jakob Ribbing RE: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Matt Hutmacher RE: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Leonid Gibiansky Re: Confidence intervals of PsN bootstrap output
Jul 11, 2011 Stephen Duffull RE: Confidence intervals of PsN bootstrap output
Jul 12, 2011 Jakob Ribbing RE: Confidence intervals of PsN bootstrap output
Jul 12, 2011 Matt Hutmacher RE: Confidence intervals of PsN bootstrap output