Re: Condition number

On 11/29/2022 7:59 PM, Ken Kowalski wrote: > Hi Matt, > > I’m pretty sure Stu Beal told me many years ago that NONMEM calculates the eigenvalues from the correlation matrix. Maybe Bob Bauer can chime in here? > > Ken > > *From:*Matthew Fidler [mailto:[email protected]] > *Sent:* Tuesday, November 29, 2022 7:56 PM > *To:* Ken Kowalski <[email protected]> > > *Cc:* Kyun-Seop Bae < [email protected] >; [email protected] ; Jeroen Elassaiss-Schaap (PD-value B.V.) < [email protected] > > > *Subject:* Re: [NMusers] Condition number > > Hi Ken, > > I am unsure, since I don't have my NONMEM manual handy. > > I based my understanding on reading about condition numbers in numerical analysis, which seemed to use the parameter estimates: > > https://en.wikipedia.org/wiki/Condition_number < https://en.wikipedia.org/wiki/Condition_number > > > If it uses the correlation matrix, it could be less sensitive. > > Matt > > On Tue, Nov 29, 2022 at 6:11 PM Ken Kowalski < [email protected] < mailto: [email protected] >> wrote: > > Hi Matt, > > Correct me if I’m wrong but I thought NONMEM calculates the > condition number based on the correlation matrix of the parameter > estimates so it is scaled based on the standard errors of the estimates. > > Ken > > *From:*Matthew Fidler [mailto:[email protected] > <mailto:[email protected]>] > *Sent:* Tuesday, November 29, 2022 7:04 PM > *To:* Ken Kowalski <[email protected] > <mailto:[email protected]>> > *Cc:* Kyun-Seop Bae <[email protected] > <mailto:[email protected]>>; [email protected] > <mailto:[email protected]>; Jeroen Elassaiss-Schaap (PD-value > B.V.) <[email protected] <mailto:[email protected]>> > *Subject:* Re: [NMusers] Condition number > > Hi Ken & Kyun-Seop, > > I agree it should be taught, since it is prevalent in the industry, > and should be looked at as something to investigate further, but no > hard and fast rule should be applied to if the model is reasonable > and fit for purpose. That should be done in conjunction with other > diagnostic plots. > > One thing that has always bothered me about the condition number is > that it is calculated based on the final parameter estimates, but > not the scaled parameter estimates. Truly the scaling is supposed > to help make the gradient on a comparable scale and fix many > numerical problems here. Hence, if the scaling works as it is > supposed to, small changes may not affect the colinearity as > strongly as the calculated condition number suggests. > > This is mainly why I see it as a number to keep in mind instead of a > hard and fast rule. > > Matt > > On Tue, Nov 29, 2022 at 5:09 PM Ken Kowalski <[email protected] > <mailto:[email protected]>> wrote: > > Hi Kyun-Seop, > > I would state things a little differently rather than say > “devalue condition number and multi-collinearity” we should > treat CN as a diagnostic and rules such as CN>1000 should NOT be > used as a hard and fast rule to reject a model. I agree with > Jeroen that we should understand the implications of a high CN > and the impact multi-collinearity may have on the model > estimation and that there are other diagnostics such as > correlations, variance inflation factors (VIF), standard errors, > CIs, etc. that can also help with our understanding of the > effects of multi-collinearity and its implications for model > development. > > That being said, if you have a model with a high CN and the > model converges with realistic point estimates and reasonable > standard errors then it may still be reasonable to accept that > model. However, in this setting I would probably still want to > re-run the model with different starting values and make sure it > converges to the same OFV and set of point estimates. > > As the smallest eigenvalue goes to 0 and the CN goes to infinity > we end up with a singular Hessian matrix (R matrix) so we know > that at some point a high enough CN will result in convergence > and COV step failures. Thus, you shouldn’t simply dismiss CN as > not having any diagnostic value, just don’t apply it in a rule > such as CN>1000 to blindly reject a model. The CN>1000 rule > should only be used to call your attention to the potential for > an issue that warrants further investigation before accepting > the model or deciding how to alter the model to improve > stability in the estimation. > > Best, > > Ken > > Kenneth G. Kowalski > > Kowalski PMetrics Consulting, LLC > > Email: [email protected] <mailto:[email protected]> > > Cell: 248-207-5082 > > *From:*[email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] *On Behalf Of *Kyun-Seop Bae > *Sent:* Tuesday, November 29, 2022 5:10 PM > *To:* [email protected] <mailto:[email protected]> > *Subject:* Fwd: [NMusers] Condition numbera > > Dear All, > > I would like to devalue condition number and multi-collinearity > in nonlinear regression. > > The reason we consider condition number (or multi-collinearity) > is that this may cause the following fitting (estimation) problems; > > 1. Fitting failure (fail to converge, fail to minimize) > 2. Unrealistic point estimates > 3. Too wide standard errors > > If you do not see the above problems (i.e., no estimation > problem with modest standard error), you do not need to give > attention to the condition number. > > I think I saw 10^(n – parameters) criterion in an old version of > Gabrielsson’s book many years ago (but not in the latest version). > > Best regards, > > Kyun-Seop Bae > > On Tue, 29 Nov 2022 at 22:59, Ayyappa Chaturvedula > <[email protected] <mailto:[email protected]>> wrote: > > Dear all, > I am wondering if someone can provide references for the > condition number thresholds we are seeing (<1000) etc. Also, > the other way I have seen when I was in graduate school that > condition number <10^n (n- number of parameters) is OK. > Personally, I am depending on correlation matrix rather than > condition number and have seen cases where condition number > is large (according to 1000 rule but less than 10^n rule) > but correlation matrix is fine. > > I want to provide these for my teaching purposes and any > help is greatly appreciated. > > Regards, > Ayyappa > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient > > Virus-free.www.avast.com > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient