RE: Right skewness in bootstrap distribution

From: [email protected] [mailto:[email protected]] On Behalf Of Sebastian Frechen Sent: den 25 april 2013 01:34 To: Felipe Hurtado; [email protected] Subject: AW: [NMusers] Right skewness in bootstrap distribution Dear All, let me also add some more thoughts. I do not think that it is a matter of a low value of the bootstrap itself considering a sample size of only 8 subjects. In this case, every estimate of a parameter and its related variability have to be treated with caution - that's clear. However, here is a very simple example that the bootstrap also may work for small sample sizes from a pure mathematical point of view: Let's say, we would like to estimate the true average weight in a population of interest. Our sample consists of n=8 observations, for example: X = (64, 68 ,72 ,75 ,76 ,83 ,91 ,93). Of course, the mean is a reasonable estimate (77.8) and we easily obtain a (slightly biased [<5%]) estimated standard error of sd(X)/sqrt(n)=3.7 Now let's run a bootstrap with 1000 samples by resampling X with replacement; I estimated in this case a standard error of 3.6 given by the standard deviation of my estimates for the mean in each bootstrap sample. At least in this example, this value corresponds to the value obtained by the ordinary method. The same applies for estimated confidence intervals. In our case, I would assume an overall normal distribution for weight. Thus, computing for the mean an exact 95%-CI based on the t-distribution with 7 degrees of freedom, we obtain (69.0 ; 86.5). Constructing a 95%-CI based on the percentiles of my bootstrap sample distribution, the CI is (70.8 ; 84.9) - slightly too narrow but still not too bad and still giving us a reasonable impression for the precision. Of course, this is a very easy example where the bootstrap does not really make sense since exact methods exist and also there may be constellations where the bootstrap does not produce sound results at all. It's clear that larger sample sizes significantly improve the bootstrap. But still, in my opinion especially in cases of small numbers, a bootstrap analysis might be even of greater value than results from asymptotic computations. Best regards, Sebastian ---------------------------------------------------------------------------- -------- Dr. Sebastian Frechen Department of Pharmacology, Clinical Pharmacology Cologne University Hospital ----------------------- Gleueler Str. 24 50931 Cologne Germany _____ Von: De Ridder, Filip [JRDBE] [[email protected]] Gesendet: Mittwoch, 24. April 2013 09:54 An: Sebastian Frechen; Felipe Hurtado; [email protected] Betreff: RE: [NMusers] Right skewness in bootstrap distribution Dear All, A few more additional thoughts regarding the use of the bootstrap here. Although I agree with what Sebastian writes, I think the value of the bootstrap is very limited when you have a sample size of only 8 subjects. Re-sampling with replacement from such a small number can easily result in replicates in which a certain subjects appears 2 or 3 even three times. In this way, the bootstrap distribution becomes quite discrete in nature and can easily start to look weird. I suspect the “subpopulation” Felipe mentions corresponds to bootstrap samples in which a subject with an extreme V1 occurs repeatedly. On a related note, did all of your bootstrap replicates yield a converged NONMEM run? Kind regards, Filip De Ridder Model-based Drug Development Janssen R&D From: [email protected] [mailto:[email protected]] On Behalf Of Sebastian Frechen Sent: Wednesday, 24 April 2013 1:29 AM To: Felipe Hurtado; [email protected] Subject: AW: [NMusers] Right skewness in bootstrap distribution Dear Felipe, I totally argree with Jakob. Maybe just some more comments on the bootstrap to support this. Each estimator comes up with its own sampling distribution reflecting the uncertainty for the obtained estimate given your data and model. You can do assumption on this distribution, for example the arithemtic mean as an estimate for the "true average in a population" follows in general a normal distribution if the sample size is suffiently large enough. However, this does not apply for every estimator! One of the basic ideas of the bootstrap is now that you do not know the underlying sampling distribution of your parameter estimate. But using the non-parametric bootstrap method (sample from you dataset with replacement), you construct this distribution (and it is not necessarily Gaussian) by estimating your parameter in each of the generated sample. This in turn gives you a fairly good feeling of how precise your estimate is given your model and the sample size. With respect to your volume: Have you tried fitting the data to one- or two-compartment models? How does the volume behave then? Why are you using a three-compartment model? Best regards, Sebastian