var-cov matrix issue?

________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Tuesday, February 24, 2009 2:09 PM To: [email protected] Cc: [email protected] Subject: Re: [NMusers] var-cov matrix issue? Hi Justin, only ETA was estimated with high SE but, again, I guess it came back to the question: how trustful it is if such error message appears ________________________________ From: "[email protected]" <[email protected]> To: [email protected] Sent: Tuesday, February 24, 2009 1:19:17 PM Subject: Fw: [NMusers] var-cov matrix issue? Dear Ethan, Algorithmically singular matrices are often a sign that that your model is ill-conditioned in some way; I would be careful in how I used the variance-covariance matrix in this scenario, and especially for simulation. Are there any parameters that are being estimated with particularly high standard errors? This might suggest overparamaterization. Not sure how helpful this is! Best Justin Justin Wilkins Senior Modeler Modeling & Simulation (Pharmacology) CHBS, WSJ-027.6.076 Novartis Pharma AG Lichtstrasse 35 CH-4056 Basel Switzerland Phone: +41 61 324 6549 Fax: +41 61 324 3039 Cell: +41 76 561 0949 Email : [email protected] <mailto:[email protected]> ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- Ethan Wu <[email protected]> Sent by: [email protected] 2009/02/24 07:12 PM To [email protected] cc Subject [NMusers] var-cov matrix issue? Dear all, I recently encounter this error message (below). My objective was to use nonmem var-cov output for approximation of distribution of parameters for performing a simulation. if such error message occur, is the var-cov matrix still OK to use? -- I know that better way to figure out distribution of parameters is to do bootstrap, but given limited time I have..... thanks "0MINIMIZATION SUCCESSFUL NO. OF FUNCTION EVALUATIONS USED: 331 NO. OF SIG. DIGITS IN FINAL EST.: 3.3 ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. ETABAR: 0.11E-02 SE: 0.23E-01 P VAL.: 0.96E+00 0S MATRIX ALGORITHMICALLY SINGULAR 0S MATRIX IS OUTPUT 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S 1 "

________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Tuesday, February 24, 2009 11:09 AM To: [email protected] Cc: [email protected] Subject: Re: [NMusers] var-cov matrix issue? Hi Justin, only ETA was estimated with high SE but, again, I guess it came back to the question: how trustful it is if such error message appears ________________________________ From: "[email protected]" <[email protected]> To: [email protected] Sent: Tuesday, February 24, 2009 1:19:17 PM Subject: Fw: [NMusers] var-cov matrix issue? Dear Ethan, Algorithmically singular matrices are often a sign that that your model is ill-conditioned in some way; I would be careful in how I used the variance-covariance matrix in this scenario, and especially for simulation. Are there any parameters that are being estimated with particularly high standard errors? This might suggest overparamaterization. Not sure how helpful this is! Best Justin Justin Wilkins Senior Modeler Modeling & Simulation (Pharmacology) CHBS, WSJ-027.6.076 Novartis Pharma AG Lichtstrasse 35 CH-4056 Basel Switzerland Phone: +41 61 324 6549 Fax: +41 61 324 3039 Cell: +41 76 561 0949 Email : [email protected] <mailto:[email protected]> ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- Ethan Wu <[email protected]> Sent by: [email protected] 2009/02/24 07:12 PM To [email protected] cc Subject [NMusers] var-cov matrix issue? Dear all, I recently encounter this error message (below). My objective was to use nonmem var-cov output for approximation of distribution of parameters for performing a simulation. if such error message occur, is the var-cov matrix still OK to use? -- I know that better way to figure out distribution of parameters is to do bootstrap, but given limited time I have..... thanks "0MINIMIZATION SUCCESSFUL NO. OF FUNCTION EVALUATIONS USED: 331 NO. OF SIG. DIGITS IN FINAL EST.: 3.3 ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. ETABAR: 0.11E-02 SE: 0.23E-01 P VAL.: 0.96E+00 0S MATRIX ALGORITHMICALLY SINGULAR 0S MATRIX IS OUTPUT 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S 1 "

-----Original Message----- From: [email protected] [mailto:[email protected]]on Behalf Of Leonid Gibiansky Sent: Tuesday, February 24, 2009 15:59 PM To: Bachman, William Cc: Ethan Wu; [email protected]; [email protected] Subject: Re: [NMusers] var-cov matrix issue? According to the manual, covariance matrix IS calculated by the default method (Rinv S Rinv) even when S is singular but the inverse covariance matrix (R Sinv R) cannot be computed as usual since S is singular (see below). From the same manual "An error message stating that the S matrix is singular indicates strong overparameterization". If some of your OMEGAs are estimated with large error, I would try to remove those ETAs from the model. Scatter plot matrix of ETAs vs ETAs could be helpful: if some of your ETAs are redundant, you could see strong correlation of the ETAs estimates. -- The inverse variance-covariance matrix R*Sinv*R is also output (labeled as the Inverse Covariance Matrix), where Sinv is the inverse of the S matrix. If S is judged to be singular, a pseudo-inverse of S is used, and since a pseudo-inverse is not unique, the inverse variance-covariance matrix is really not unique. In either case, the inverse variance-covariance matrix can be used to develop a joint con- fidence region for the complete set of population parameters. As we usually develop a confidence region for a very limited set of popula- tion parameters, this use of the inverse variance-covariance matrix is somewhat limited. -- -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Bachman, William wrote: > As a clarification, this is not an error. It is an indication of a > numerical condition generated by the matrix algebra. it says that the > covariance could not be calculated by the default method (possibly due > to ill conditioning) so it was calculated by an alternative method. You > could generate standard errors by an alternative method, e.g. bootstrap, > and compare them to those produced by NONMEM to make your decision to > trust or not trust the values. > > ------------------------------------------------------------------------ > *From:* [email protected] > [mailto:[email protected]] *On Behalf Of *Ethan Wu > *Sent:* Tuesday, February 24, 2009 2:09 PM > *To:* [email protected] > *Cc:* [email protected] > *Subject:* Re: [NMusers] var-cov matrix issue? > > Hi Justin, only ETA was estimated with high SE > but, again, I guess it came back to the question: how trustful it is if > such error message appears > > ------------------------------------------------------------------------ > *From:* "[email protected]" <[email protected]> > *To:* [email protected] > *Sent:* Tuesday, February 24, 2009 1:19:17 PM > *Subject:* Fw: [NMusers] var-cov matrix issue? > > > Dear Ethan, > > Algorithmically singular matrices are often a sign that that your model > is ill-conditioned in some way; I would be careful in how I used the > variance-covariance matrix in this scenario, and especially for > simulation. Are there any parameters that are being estimated with > particularly high standard errors? This might suggest overparamaterization. > > Not sure how helpful this is! > > Best > Justin > *Justin Wilkins > Senior Modeler** > Modeling & Simulation (Pharmacology)* > CHBS, WSJ-027.6.076 > Novartis Pharma AG > Lichtstrasse 35 > CH-4056 Basel > Switzerland > Phone: +41 61 324 6549 > Fax: +41 61 324 3039 > Cell: +41 76 561 0949 > Email : [email protected]_ <mailto:[email protected]> > > > > ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- > *Ethan Wu <[email protected]>* > Sent by: [email protected] > > 2009/02/24 07:12 PM > > > To > [email protected] > cc > > Subject > [NMusers] var-cov matrix issue? > > > > > > > > > Dear all, > I recently encounter this error message (below). My objective was to > use nonmem var-cov output for approximation of distribution of > parameters for performing a simulation. > if such error message occur, is the var-cov matrix still OK to use? > -- I know that better way to figure out distribution of parameters is to > do bootstrap, but given limited time I have..... > > thanks > > "0MINIMIZATION SUCCESSFUL > NO. OF FUNCTION EVALUATIONS USED: 331 > NO. OF SIG. DIGITS IN FINAL EST.: 3.3 > ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, > AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. > ETABAR: 0.11E-02 > SE: 0.23E-01 > P VAL.: 0.96E+00 > 0S MATRIX ALGORITHMICALLY SINGULAR > 0S MATRIX IS OUTPUT > 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S > 1 > " > >

________________________________ From: Bob Leary <[email protected]> To: [email protected] Sent: Wednesday, February 25, 2009 8:33:53 AM Subject: RE: [NMusers] var-cov matrix issue? If S is singular, then the 'covariance' matrix Rinv * S * Rinv is also singular, as is the 'inverse coveriance matrix' R*Spseudoinv*R (the eigenvalues of Spseudoinv for the usual Moore Penrose pseudoinverse are the inverse of the eigenvalues of S, except where the S has a zero eigenvalue, in which case the corresponding eigenvalue of Spseudoinv is also zero. The eigenvectors are the same for S and Spseudoinv). Thus none of these quantities is really directly suitable for use in simulation if positive definiteness is a requirement. Robert H. Leary, PhD Fellow Pharsight - A Certara(tm) Company 5625 Dillard Dr., Suite 205 Cary, NC 27511 Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871 Email: [email protected] > This email message (including any attachments) is for the sole use of the > intended recipient and may contain confidential and proprietary > information. Any disclosure or distribution to third parties that is not > specifically authorized by the sender is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy all > copies of the original message. -----Original Message----- From: [email protected] [mailto:[email protected]]on Behalf Of Leonid Gibiansky Sent: Tuesday, February 24, 2009 15:59 PM To: Bachman, William Cc: Ethan Wu; [email protected]; [email protected] Subject: Re: [NMusers] var-cov matrix issue? According to the manual, covariance matrix IS calculated by the default method (Rinv S Rinv) even when S is singular but the inverse covariance matrix (R Sinv R) cannot be computed as usual since S is singular (see below). From the same manual "An error message stating that the S matrix is singular indicates strong overparameterization". If some of your OMEGAs are estimated with large error, I would try to remove those ETAs from the model. Scatter plot matrix of ETAs vs ETAs could be helpful: if some of your ETAs are redundant, you could see strong correlation of the ETAs estimates. -- The inverse variance-covariance matrix R*Sinv*R is also output (labeled as the Inverse Covariance Matrix), where Sinv is the inverse of the S matrix.. If S is judged to be singular, a pseudo-inverse of S is used, and since a pseudo-inverse is not unique, the inverse variance-covariance matrix is really not unique. In either case, the inverse variance-covariance matrix can be used to develop a joint con- fidence region for the complete set of population parameters. As we usually develop a confidence region for a very limited set of popula- tion parameters, this use of the inverse variance-covariance matrix is somewhat limited. -- -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 Bachman, William wrote: > As a clarification, this is not an error. It is an indication of a > numerical condition generated by the matrix algebra. it says that the > covariance could not be calculated by the default method (possibly due > to ill conditioning) so it was calculated by an alternative method. You > could generate standard errors by an alternative method, e.g. bootstrap, > and compare them to those produced by NONMEM to make your decision to > trust or not trust the values. > > ------------------------------------------------------------------------ > *From:* [email protected] > [mailto:[email protected]] *On Behalf Of *Ethan Wu > *Sent:* Tuesday, February 24, 2009 2:09 PM > *To:* [email protected] > *Cc:* [email protected] > *Subject:* Re: [NMusers] var-cov matrix issue? > > Hi Justin, only ETA was estimated with high SE > but, again, I guess it came back to the question: how trustful it is if > such error message appears > > ------------------------------------------------------------------------ > *From:* "[email protected]" <[email protected]> > *To:* [email protected] > *Sent:* Tuesday, February 24, 2009 1:19:17 PM > *Subject:* Fw: [NMusers] var-cov matrix issue? > > > Dear Ethan, > > Algorithmically singular matrices are often a sign that that your model > is ill-conditioned in some way; I would be careful in how I used the > variance-covariance matrix in this scenario, and especially for > simulation. Are there any parameters that are being estimated with > particularly high standard errors? This might suggest overparamaterization. > > Not sure how helpful this is! > > Best > Justin > *Justin Wilkins > Senior Modeler** > Modeling & Simulation (Pharmacology)* > CHBS, WSJ-027.6.076 > Novartis Pharma AG > Lichtstrasse 35 > CH-4056 Basel > Switzerland > Phone: +41 61 324 6549 > Fax: +41 61 324 3039 > Cell: +41 76 561 0949 > Email : [email protected]_ <mailto:[email protected]> > > > > ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- > *Ethan Wu <[email protected]>* > Sent by: [email protected] > > 2009/02/24 07:12 PM > > > To > [email protected] > cc > > Subject > [NMusers] var-cov matrix issue? > > > > > > > > > Dear all, > I recently encounter this error message (below). My objective was to > use nonmem var-cov output for approximation of distribution of > parameters for performing a simulation. > if such error message occur, is the var-cov matrix still OK to use? > -- I know that better way to figure out distribution of parameters is to > do bootstrap, but given limited time I have..... > > thanks > > "0MINIMIZATION SUCCESSFUL > NO. OF FUNCTION EVALUATIONS USED: 331 > NO. OF SIG. DIGITS IN FINAL EST.: 3.3 > ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, > AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. > ETABAR: 0..11E-02 > SE: 0.23E-01 > P VAL.: 0.96E+00 > 0S MATRIX ALGORITHMICALLY SINGULAR > 0S MATRIX IS OUTPUT > 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S > 1 > " > >

________________________________ From: "[email protected]" <[email protected]> To: Ethan Wu <[email protected]> Cc: [email protected] Sent: Tuesday, February 24, 2009 3:07:30 PM Subject: RE: [NMusers] var-cov matrix issue? Hi, Ethan, I think your question can be reduced whether pseudo-inverse matrix can be used instead of inverse matrix. I do not know quite different cases, but I suppose it can be used. To be more adequate answer in your context, MATRIX=R option could be more appropriate, if you use VAR-COV matrix output for simulation under normal distribution assumtion, If your data supports normal distribution assumption, MATRIX=R option will not give much difference in SEs. Default VAR-COV output in NONMEM is a kind of sandwich estimate, which is thought to be more robust (a little larger) than inverse Fisher's information matrix (given MATRIX=R option). Some caution is necessary to simulate omega matrix that is alwasy positive definite. This may help you.. Thanks, Kyun Seop Bae MD PhD Email: [email protected] ________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Tuesday, February 24, 2009 11:09 AM To: [email protected] Cc: [email protected] Subject: Re: [NMusers] var-cov matrix issue? Hi Justin, only ETA was estimated with high SE but, again, I guess it came back to the question: how trustful it is if such error message appears ________________________________ From: "[email protected]" <[email protected]> To: [email protected] Sent: Tuesday, February 24, 2009 1:19:17 PM Subject: Fw: [NMusers] var-cov matrix issue? Dear Ethan, Algorithmically singular matrices are often a sign that that your model is ill-conditioned in some way; I would be careful in how I used the variance-covariance matrix in this scenario, and especially for simulation. Are there any parameters that are being estimated with particularly high standard errors? This might suggest overparamaterization. Not sure how helpful this is! Best Justin Justin Wilkins Senior Modeler Modeling & Simulation (Pharmacology) CHBS, WSJ-027.6.076 Novartis Pharma AG Lichtstrasse 35 CH-4056 Basel Switzerland Phone: +41 61 324 6549 Fax: +41 61 324 3039 Cell: +41 76 561 0949 Email : [email protected] <mailto:[email protected]> ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- Ethan Wu <[email protected]> Sent by: [email protected] 2009/02/24 07:12 PM To [email protected] cc Subject [NMusers] var-cov matrix issue? Dear all, I recently encounter this error message (below). My objective was to use nonmem var-cov output for approximation of distribution of parameters for performing a simulation. if such error message occur, is the var-cov matrix still OK to use? -- I know that better way to figure out distribution of parameters is to do bootstrap, but given limited time I have..... thanks "0MINIMIZATION SUCCESSFUL NO. OF FUNCTION EVALUATIONS USED: 331 NO. OF SIG. DIGITS IN FINAL EST.: 3.3 ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. ETABAR: 0.11E-02 SE: 0.23E-01 P VAL.: 0.96E+00 0S MATRIX ALGORITHMICALLY SINGULAR 0S MATRIX IS OUTPUT 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S 1 "

________________________________ From: "[email protected]" <[email protected]> To: Ethan Wu <[email protected]> Sent: Thursday, February 26, 2009 8:19:06 PM Subject: RE: [NMusers] var-cov matrix issue? Dear Ethan, If your model has only diagonal elements in OMEGA matrix, you don't need to care about the following - positive definiteness of OMEGA matrix during simulation. If you use VAR-COV output of NONMEM for the simulation, it means you generate THETAs, OMEGA matrix and SIGMA matrix from the MVN distribution of VAR-COV matrix. OMEGA and SIGMA matrix are always positive definite in nature. However, if you generate full-block OMEGA matrix using MVN (multi-variate normal) of VAR-COV matrix, many of generated OMEGA matrix will not be positive definite. One way to avoid this is test positive definiteness and discard in-adequate ones. This may help you. Regards, Kyun Seop Bae MD PhD ________________________________ From: Ethan Wu [mailto:[email protected]] Sent: Thursday, February 26, 2009 4:05 AM To: [email protected] Cc: [email protected] Subject: Re: [NMusers] var-cov matrix issue? Dear Kyun, thanks for your help. I don't know if I understand this one "Some caution is necessary to simulate omega matrix that is alwasy positive definite. " Could you explain a bit more? ________________________________ From: "[email protected]" <[email protected]> To: Ethan Wu <[email protected]> Cc: [email protected] Sent: Tuesday, February 24, 2009 3:07:30 PM Subject: RE: [NMusers] var-cov matrix issue? Hi, Ethan, I think your question can be reduced whether pseudo-inverse matrix can be used instead of inverse matrix. I do not know quite different cases, but I suppose it can be used. To be more adequate answer in your context, MATRIX=R option could be more appropriate, if you use VAR-COV matrix output for simulation under normal distribution assumtion, If your data supports normal distribution assumption, MATRIX=R option will not give much difference in SEs. Default VAR-COV output in NONMEM is a kind of sandwich estimate, which is thought to be more robust (a little larger) than inverse Fisher's information matrix (given MATRIX=R option). Some caution is necessary to simulate omega matrix that is alwasy positive definite. This may help you. Thanks, Kyun Seop Bae MD PhD Email: [email protected] ________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Ethan Wu Sent: Tuesday, February 24, 2009 11:09 AM To: [email protected] Cc: [email protected] Subject: Re: [NMusers] var-cov matrix issue? Hi Justin, only ETA was estimated with high SE but, again, I guess it came back to the question: how trustful it is if such error message appears ________________________________ From: "[email protected]" <[email protected]> To: [email protected] Sent: Tuesday, February 24, 2009 1:19:17 PM Subject: Fw: [NMusers] var-cov matrix issue? Dear Ethan, Algorithmically singular matrices are often a sign that that your model is ill-conditioned in some way; I would be careful in how I used the variance-covariance matrix in this scenario, and especially for simulation. Are there any parameters that are being estimated with particularly high standard errors? This might suggest overparamaterization. Not sure how helpful this is! Best Justin Justin Wilkins Senior Modeler Modeling & Simulation (Pharmacology) CHBS, WSJ-027.6.076 Novartis Pharma AG Lichtstrasse 35 CH-4056 Basel Switzerland Phone: +41 61 324 6549 Fax: +41 61 324 3039 Cell: +41 76 561 0949 Email : [email protected] <mailto:[email protected]> ----- Forwarded by Justin Wilkins/PH/Novartis on 2009/02/24 07:15 PM ----- Ethan Wu <[email protected]> Sent by: [email protected] 2009/02/24 07:12 PM To [email protected] cc Subject [NMusers] var-cov matrix issue? Dear all, I recently encounter this error message (below). My objective was to use nonmem var-cov output for approximation of distribution of parameters for performing a simulation. if such error message occur, is the var-cov matrix still OK to use? -- I know that better way to figure out distribution of parameters is to do bootstrap, but given limited time I have..... thanks "0MINIMIZATION SUCCESSFUL NO. OF FUNCTION EVALUATIONS USED: 331 NO. OF SIG. DIGITS IN FINAL EST.: 3.3 ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0.. ETABAR: 0.11E-02 SE: 0.23E-01 P VAL.: 0.96E+00 0S MATRIX ALGORITHMICALLY SINGULAR 0S MATRIX IS OUTPUT 0INVERSE COVARIANCE MATRIX SET TO RS*R, WHERE S* IS A PSEUDO INVERSE OF S 1 "