Re: comparing theta's

From: "Rik Schoemaker" <RS@chdr.nl> Date: Mon, 29 Mar 1999 11:27:03 +0200 Subject: Re: comparing theta's In reply to Nick's reply, I think you should be extremely cautious in using empirical Bayes estimates for subsequent statistical comparisons whether it be using parametric or non-parametric tests. The reasons have to do with distribution of information and shrinkage. Empirical Bayes estimates are a composite of individual and population information which makes them dependent. To illustrate the point, think of an extreme example with one subject with a very well defined curve and sixty subjects contributing only one or two points. This will result in 61 empirical Bayes estimates which will be closely clustered around the one well defined subject (due to shrinkage). If you were to use these 61 estimates you (and your statistical analysis) will have no clue which estimate is 'precise' and which estimates are not and therefore all will be considered equally precise (distribution of information). Now if you were to compare this model with another one, once again with one good and 60 imprecise subjects, you're bound to arrive at statistical significance because it looks like you have two times 61 closely clustered estimates at (most likely) different locations. Real life problems will never be as extreme but there will always be some distribution of information and shrinkage. You can use empirical Bayes estimates for all sorts of diagnostics and it may be compellingly argued that the emprical Bayes estimates are the 'best' estimates for an individual. Just make sure you don't fool yourself by using them for statistical testing. Regards, Rik Schoemaker CHDR, NL