RE: Bootstrap in combination with PRIOR?
Jacob,
To address your question, I would compare the uncertainty in models parameters
obtained after running a non-parametric bootstrap (NP-BS) with that obtained
after a parametric bootstrap (P-BS).
P-BS allows you to understand (under the null hypothesis) the expected
posterior distribution of model parameters given the design of the new study
and, therefore, you can assess if your new data would contain enough
information to speak about certain parameters.
For instance, if after running a P-BS, you get the same estimate of a certain
parameter in all bootstraps replicates, it means the data available does not
speak about that parameter (regardless of how strong the prior is) and,
consequently, you can fix it before running the PRIOR and, of course, cannot
derive any new conclusion on that parameter based on the new data.
If the new data speaks substantially about model parameters (because an
informative design and/or large sample size), the P-BS would provide you with
the expected value (and uncertainty) of the model parameters (assuming the new
data are arising from the same population used to obtain the priors) and will
give you an idea about the contribution of the new study design in reducing the
uncertainty in model parameter from the priors.
However, if the new data are not arising from the same population used to
obtain the priors, the uncertainty in model parameters might not be reduced
and, therefore, you need to make a decision about the appropriateness of using
PRIOR in that situation. You can make that assessment by comparing the point
estimates (and uncertainty) of the updated parameters with the results of the
P-BS. If the updated parameter estimates falls within the expected updated
values of the model parameters, then you can assume the new data are arising
from the same population used to obtain the priors, provided you have enough
power. However, this exercise tends to be conservative and additional
simulation/estimation exercise might be needed.
Hope it helps. Enjoy the Holiday Season!
Juan J. Perez-Ruixo.
Quoted reply history
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Leonid Gibiansky
Sent: jueves, 22 de diciembre de 2011 7:08
To: Brogren Jacob
Cc: [email protected]
Subject: Re: [NMusers] Bootstrap in combination with PRIOR?
Jacob,
Bootstrap procedure vary only the new data. The information in priors is
still the same. Therefore, results will be as dependent on the priors as
the parameter estimates of the final model. When the priors are strong
(informative) it would require a lot of new data to move the parameters
of the final model and the parameter estimates of each of the bootstrap
samples. In the extreme case of the very strong priors (e.g., when the
parameters are simply fixed) parameters of all bootstrap samples will
also be fixed at the same values. Since the results are dependent of the
strength (informative content) of priors, I doubt that any conclusions
can be made about each particular case based on the other-people
examples. This is likely to be decided on the case-by-case basis.
Nonmem standard errors are usually in a good agreement with the
bootstrap results. Therefore, influence of the priors on the parameter
estimates (precision) can be evaluated using the standard errors (much
quicker than running bootstrap multiple times with different prior
values). Extreme test would be to run the model without priors, and see
how this would influence the confidence intervals.
Regards,
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
On 12/22/2011 7:58 AM, Brogren Jacob wrote:
> Dear nmusers,
>
> Merry Holiday! (you may change the variable 'Holiday' to an arbitrary value)
>
> First, accept my humble excuse - I can't share any code for this example.
>
> I'm assessing a model where $PRIOR is used to stabilize the fit of an
> 'old' model to sparse data using NONMEM. It seems to be the only way
> forward. Using the final model the Applicant has then performed a
> non-parametric bootstrap (N=1000) to investigate i.a. robustness of the
> model parameter estimates.
>
> From a non-scientific literature search (Google Scholar "NONMEM + PRIOR
> + Bootstrap") some examples show up where Bootstrap and PRIOR has been
> combined.
>
> I'm curious about if any nm-user has investigated how much and in what
> way the prior distribution of parameters affect the bootstrap estimates
> (eg. 2.5^th , 50^th and 97.5^th percentiles)? I would guess that the
> stronger the prior the more it would affect the outcome of a bootstrap.
> Or is that a misconception?
>
> Cheers
>
> Jacob
>
> LV_Mac3
>
> Medical Products Agency
>
> Jacob Brogren
> Assessor
> Efficacy and Safety 2
>
>
>
> P.O. Box 26, SE-751 03 Uppsala, Sweden
> Visiting address: Dag Hammarskjölds väg 42
> Phone: + 46 (0) 18 17 47 64, switchboard: + 46 (0) 18 17 46 00
> Mobile: + 46 (0) xxx xx xx, Fax: + 46 (0) 18 54 85 66
> www.lakemedelsverket.se http://www.lakemedelsverket.se
>