Group:
I have bootstrap analysis (my first) parameter estimates and model
parameters. The PDxPOP/NONMEM manual I have does not provide
any guidance as to how I can statistically compare these two (or do I
need to?). I also have histograms for the thetas in bootstrap analysis.
I can make some visual judgements but is there a way to statistically
compare the two results (bootstrap v model) built in to the NONMEM
that I can use to quickly get some statistical comparison results?
How else can I use the bootstrap results to confirm the fact that the
model I have is the best fit for the data?
Thanks in advance.
Varsha Mehta, MS(CRDSA), Pharm.D., FCCP
Clinical Associate Professor
Pharmacy, Pediatrics and Communicable Diseases
Clinical Pharmacist Neonatal Critical Care
University of Michigan
(O) 734-936-8985
(F) 734-936-6946
[email protected]
**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be
used for urgent or sensitive issues
Bootstrap analysis
Varsha,
Congratulations on discovering how to use a bootstrap to evaluate the distribution of your model parameter estimates.
The bootstrap mean is probably a more robust estimate of the true value of the parameter than the value estimated from the original data. I prefer to report the bootstrap mean for this reason.
The uncertainty, e.g. 95% confidence interval, can sometimes be useful for model evaluation but more commonly is is best used to keep journal reviewers 'happy'. There are very few other real applications of knowing the uncertainty of a single parameter but it might be used to try to demonstrate that a PD parameter (e.g. Emax) is different from zero and thus indicate that the drug does something useful.
The good news is that you don't have to worry about using bootstraps "to confirm the fact that the model I have is the best fit for the data". The bootstrap can never confirm this for you. You need to buy a subscription to 'Talk to God' in order to get that kind of information.
Nick
Varsha Mehta wrote:
> Group:
>
> I have bootstrap analysis (my first) parameter estimates and model
> parameters. The PDxPOP/NONMEM manual I have does not provide
> any guidance as to how I can statistically compare these two (or do I
> need to?). I also have histograms for the thetas in bootstrap analysis.
> I can make some visual judgements but is there a way to statistically
> compare the two results (bootstrap v model) built in to the NONMEM
> that I can use to quickly get some statistical comparison results?
>
> How else can I use the bootstrap results to confirm the fact that the
> model I have is the best fit for the data?
>
> Thanks in advance.
>
> Varsha Mehta, MS(CRDSA), Pharm.D., FCCP
> Clinical Associate Professor
> Pharmacy, Pediatrics and Communicable Diseases
> Clinical Pharmacist Neonatal Critical Care
> University of Michigan
> (O) 734-936-8985
> (F) 734-936-6946
> [email protected]
>
> **********************************************************
> Electronic Mail is not secure, may not be read every day, and should not be
> used for urgent or sensitive issues
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[email protected] tel:+64(9)923-6730 fax:+64(9)373-7090
mobile: +33 64 271-6369 (Apr 6-Jul 17 2009)
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
I do not think that bootstrap mean is a useful statistics. Median of the bootstrap distribution could/should be compared with the final-model point estimates. Precision of the parameter estimates can be evaluated as a 95% confidence interval defined as an interval between 2.5 and 97.5 percentiles of the bootstrap distributions. Another useful application of the bootstrap parameter estimates is the investigation of the correlation between those. A scatter-plot matrix of parameters versus parameters readily reveals existing correlations. If those are very strong, the model could/should be improved to remove over-parameterization. Histograms can be useful if you put the final-model estimate on top of those distributions. It should be somewhere close to the center of the bootstrap parameter distribution. If you like some p-value, you can compute the one-sided or two-sided probability of observing the value as extreme as the final-model parameter estimates based on the bootstrap distribution (using a percent of bootstrap estimates that are below or above the final-model. estimate).
Thanks
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Nick Holford wrote:
> Varsha,
>
> Congratulations on discovering how to use a bootstrap to evaluate the distribution of your model parameter estimates.
>
> The bootstrap mean is probably a more robust estimate of the true value of the parameter than the value estimated from the original data. I prefer to report the bootstrap mean for this reason.
>
> The uncertainty, e.g. 95% confidence interval, can sometimes be useful for model evaluation but more commonly is is best used to keep journal reviewers 'happy'. There are very few other real applications of knowing the uncertainty of a single parameter but it might be used to try to demonstrate that a PD parameter (e.g. Emax) is different from zero and thus indicate that the drug does something useful.
>
> The good news is that you don't have to worry about using bootstraps "to confirm the fact that the model I have is the best fit for the data". The bootstrap can never confirm this for you. You need to buy a subscription to 'Talk to God' in order to get that kind of information.
>
> Nick
>
> Varsha Mehta wrote:
>
> > Group:
> >
> > I have bootstrap analysis (my first) parameter estimates and model
> > parameters. The PDxPOP/NONMEM manual I have does not provide
> > any guidance as to how I can statistically compare these two (or do I
> > need to?). I also have histograms for the thetas in bootstrap analysis.
> > I can make some visual judgements but is there a way to statistically
> > compare the two results (bootstrap v model) built in to the NONMEM
> > that I can use to quickly get some statistical comparison results?
> >
> > How else can I use the bootstrap results to confirm the fact that the
> > model I have is the best fit for the data?
> >
> > Thanks in advance.
> >
> > Varsha Mehta, MS(CRDSA), Pharm.D., FCCP
> > Clinical Associate Professor
> > Pharmacy, Pediatrics and Communicable Diseases
> > Clinical Pharmacist Neonatal Critical Care
> > University of Michigan
> > (O) 734-936-8985
> > (F) 734-936-6946
> > [email protected]
> >
> > **********************************************************
> >
> > Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues