Re: RE: Slow PRED in tables?

On Mon, Feb 3, 2014, at 06:38 AM, Bob Leary wrote: > Jerry, you are correct - the temporal pattern of the WRES components > WRES(I) could be quite misleading. One simple measure of agreement might > be correlation between > WRES(I) and the actual residual RES(I). > > For the case of the Cholesky decomposition, if we have M observations, > then the 1st component WRES(1) actually just depends on the > First residual RES(1) , so there is a 100% correlation between WRES(1) > and RES(1). But WRES(2) depends on RES(1) and RES(2), and more > generally WRES(K) depends on the > First K residuals. So the correlation between RES(i) and WRES(i) gets > weaker for the later values. > > In the case of the eigen-decomposition version of WRES, each individual > WRES(I) value depend on all the RES(I) values, but there is still a > positive correlation > Between RES(I) and WRES(I). I looked at this some time ago, and I think > I think I convinced myself that the eigenvector method actually maximizes > some average measure of correlation > (or more likely squared correlation) Between RES(I) and WRES(I), but > offhand I don't remember the details and whether or not I actually was > able to prove something. > > One thing you need to be careful about is picking the whitening matrix to > be a function of the residual data RES(i). In this case the whitening > matrix becomes a random matrix and > We can not longer be sure that the resulting product of the whitening > matrix and the random residuals on which it depends is actually anything > close to normally distributed with > Independent unit variance components. For an extreme example, for any > given WRES vector, one could always multiply it > By an orthogonal Householder matrix H to get WRESNEW = H*WRES, where > H*H'=I, but H is selected to make, for example, the first component of > WRESNEW equal to a positive value and all the other > Components equal to zero. Such a selection would be close to useless as > a diagnostic. > > My own conjecture is that for each particular model and experimental > design, there is a whitening matrix which is best in some sense for > exposing model misspecification, and > That perhaps the projection pursuit methodology , in combination with a > large number of simulations from that model, could be used to find it. > Whether this is a practical idea, I don't know. > > > -----Original Message----- > From: Nedelman, Jerry [mailto:[email protected]] > Sent: Monday, February 03, 2014 8:06 AM > To: Bob Leary; Bauer, Robert; [email protected] > Subject: RE: Slow PRED in tables? > > Bobs: Thanks for these insights. I've often wondered about the use of > weighted residuals as diagnostics where the pattern of residuals against > an observation-related covariate, such as time, is examined. Each > weighted residual is a linear combination of all residuals, so it is not > attached to any unique time. > > How serious a concern is this? Is it possible that the pattern could be > misleading because of how the weights are distributed? > > Is there some way of knowing how strong the attachment is? Could the > strength of the attachment be an optimization criterion for picking the > whitening matrix? Might the choice of best whitening matrix depend on the > covariate being diagnosed? > > Thanks, > Jerry > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of Bob Leary > Sent: Wednesday, January 29, 2014 7:03 PM > To: Bauer, Robert; Eleveld, DJ; [email protected] > Subject: [NMusers] RE: Slow PRED in tables? > > Doug and Bob, > > This is point that I have long been interested in. For each individual, > the original WRES in NONMEM is based on a Schur decomposition of the > estimated covariance matrix (I have verified this numerically using > MATLAB) of the residual vector. Thus if the individual covariance matrix > of the residuals is C, and the residual vector is r, WRES is define as > C^(-1/2)*r, where C= U*diag(lambda)*U' , and the columns of U are the > eigenvectors of C, and lambda is the diagonal matrix of eigenvalues. > The main point of the computation is that the covariance of C^(-1/2)*r > is a unit matrix, where C^(-1/2) = U*lambda(^-1.2)*U' has the rough > interpretation of being a square root of C^(-1). > > Further diagnostic analysis of the components of WRES treats them as if > they were independent N(0,1) random variables since indeed are are > uncorrelated and normalized to have unit variance (although the normality > assumption is doubtful for hgihly nonlinear models). > > Thus the computation of WRES in NM7.2 requires the computation of all > eigenvalues and eigenvectors of the C matrix, which for large matrices > can start to be expensive (although the best LAPACK implementation of a > symmetric positive definite eigendecomposition uses Jim Demmel's O(N^1/3) > routine and maybe considerably faster than what you currently seeing in > NM7.2 - I have no way of knowing what NM7.2 actually uses.) > > But in any event, all the WRES computation is doing is de-correlating and > normalizing the residuals. Any matrix W such that W*r has a covariance > matrix equal to the unit matrix will do. In particular, choosing W = > lower triangular Cholesky decompostion of C^(-1), works just fine and is > much faster to compute than to compute than a Schur eigen-decompostion. > Based on the name, I assume that WRESCHOL is just WRES implemented with > such a Cholesky decomposition. > > WHich is better - WRES or WRESCHOL ? This is difficult question without > further information. It does point out that the details of the WRES > results, no matter how done, should be take with a large grain of salt > (WRESCHOL numerically will look totally different than WRES, yet they are > both equally valid de-correlation/normalizations) > > Electrical engineers have long been aware of this > de-correlation/normalization ambiguity. Basically if C(^-1/2) is any > matrix that decorrelates and normalizes the residuals, so is U*C^(-1/2), > where U is any orthogonal matrix (U'*U=I) Thus there are an infinite > number of choices to use for computing a reasonable WRES for further > diagnostic analysis. In the electrical engineering community, any matrix > of the form U*C^(-1/2) is called a whitening matrix, since it transforms > correlated data into 'whitened', decorrelated data with unit variances. > > In order to decide which decorrelation is best, one must impose an > optimization criterion to pick the particular best whitening matrix, This > is not so easy to do,but has given rise to a large literature under the > general name of 'projection pursuit' methods. > > > I have long suspected there may be something here worth looking into with > respect to population PK/PD model evaluation techniques, but his is still > an open question. > > > > > > ________________________________________ > From: [email protected] [[email protected]] On > Behalf Of Bauer, Robert [[email protected]] > Sent: Wednesday, January 29, 2014 3:34 PM > To: Eleveld, DJ; [email protected] > Subject: [NMusers] RE: Slow PRED in tables? > > Douglas: > Thank you for sharing your code and data with me. When PRED, WRES, or > RES are requested, NONMEM calls the weighted residual routine PRRES, > which calculates PRED, WRES, and RES together. > Your problem has a large number of data per subject (up to 1544) , and > evaluating WRES requires some matrix algebra on up to a 1544x1544 matrix, > which can take time to compute for each subject. The standard method of > calculating WRES in NONMEM 7.3 and lower versions is inefficient, and I > have made it more efficient for a future version of nonmem 7.4, so your > TABLE step will calculate in 15 minutes. Meantime, in recently released > NONMEM 7.3, you can use the WRESCHOL option, and the $TABLE step was > calculated in 3 minutes. > > Robert J. Bauer, Ph.D. > Vice President, Pharmacometrics, R&D > ICON Development Solutions > 7740 Milestone Parkway > Suite 150 > Hanover, MD 21076 > Tel: (215) 616-6428 > Mob: (925) 286-0769 > Email: [email protected]<mailto:[email protected]> > Web: http://www.iconplc.com/ > > From: [email protected] [mailto:[email protected]] > On Behalf Of Eleveld, DJ > Sent: Tuesday, January 28, 2014 9:19 AM > To: [email protected] > Subject: [NMusers] Slow PRED in tables? > > Hello everyone, > > I have a curious problem with slow PRED calculations in tables. The > estimations are reasonably fast, 471 seconds for 23 iterations. If there > is no PRED in any tables then NONMEM finishes a moment after the message > about elapsed estimation time. But if is PRED in a table then it takes an > extremely long time to return, I waited more than 10 hours before I broke > off the run. > > If convergence of the algorithm is reasonably fast I get the impression > that the differential equations are not problematic (it is $DES solved > with ADVAN13 TOL=9). In fact if I restart the run with all $OMEGA to (0 > FIXED) - to force all ETAs to zero - and do a MAX=0 run then it is quite > fast. > > I would expect calculating PRED would be simply setting ETA=0 and solving > the differential equations just once. Am I missing anything in how PRED > in a table is calculated? > > Has anyone seen this kind of thing before? Any advice or tips? > > Warm regards, > > Douglas Eleveld > > > ________________________________ >