RE: Confidence intervals of PsN bootstrap output
Leonid
> of them. If some realizations are so special that the model behaves in
> an unusual way (with any definition of unusual: non-convergence, not
> convergence of the covariance step, parameter estimates at the boundary,
> etc.) we either need to accept those as is, or work with each of those
> special data sets one by one to push to the parameter estimates that we
> can accept, or change the bootstrap procedure (add stratification by
> covariates, by dose level, by route of administration, etc.) so that all
> data sets behave similarly.
I agree with your comments above and there are times that this might be
possible. But in essence I hear that BS may become overtly time consuming in
some cases if individual BS'd data sets require almost a full modelling
procedure to gain appropriate convergence outcomes. This might make a long BS
into an almost impossible task.
The risk a user then runs is once you know there are runs that don't work as
intended you cannot ignore them. If you can't afford the time resource (say a
single run takes 12-24 hours) to recover each of these (say 20% of 1000 BS
runs) and since you cannot un-know the results you're stuck. Would not a
pragmatic view be appropriate? Report the outcomes of the BS runs as they
occurred but then select those runs that terminated appropriately? Otherwise
BS is too risky...
Steve
PS All models are wrong :-)
--
>
>
Quoted reply history
> On 7/10/2011 2:57 PM, Stephen Duffull wrote:
> > Nick, Jakob, Marc et al
> >
> >> Thanks for your helpful comments. I agree with you that any results that
> >> are at a boundary should be discarded from the bootstrap distribution.
> >
> > On the whole I the sentiments in this thread align with anecdotal findings
> from my experience. But, I was just wondering how you define your boundaries
> for variance and covariance parameters (e.g. OMEGA terms)?
> >
> > For variance terms, lower boundaries seems reasonably straightforward (e.g.
> 1E-5 seems close to zero). Upper boundaries are of course open, for the
> variance of a log-normal ETA would 1E+5 or 1E+4 be large enough to be
> considered close to a boundary? At what value would you discard the result?
> At what correlation value would you discard a result (>0.99,> 0.97...) as
> being close to 1. Clearly if this was for regulatory work you could define
> these a priori after having chosen any arbitrary cut-off. But the devil here
> lies with the non-regulatory work where you may not have defined these
> boundaries a priori.
> >
> > Steve
> > --
> > Professor Stephen Duffull
> > Chair of Clinical Pharmacy
> > School of Pharmacy
> > University of Otago
> > PO Box 56 Dunedin
> > New Zealand
> > E: [email protected]
> > P: +64 3 479 5044
> > F: +64 3 479 7034
> > W: http://pharmacy.otago.ac.nz/profiles/stephenduffull
> >
> > Design software: www.winpopt.com
> >
> >