Dear Leonid
Thank you for this informative response. It'll definitely help me in finding my
bearings.
Cheers
Andreas
Von: Leonid Gibiansky [mailto:[email protected]]
Gesendet: Dienstag, 7. September 2010 17:08
An: Steingötter Andreas
Cc: nmusers; Rickmer Braren
Betreff: Re: [NMusers] Negative eigenvalues, over-paramterization, finding most
sensitive parameter
Andreas,
I doubt that one can give a general answer (without looking on the specific
model) but here is my understanding of the situation:
In over-parametrized models, there is one or more degenerate directions (in the
parameter space) where changes of the parameters do not change the fit (i.e.,
where there is no data to estimate each parameter, only some combination). The
model can be well-defined in the orthogonal directions. The simplest example is
oral absorption: without IV data, F( bioavaialbility), CL and V are not
definable. However, CL/F and V/F can be estimated. This leads to two different
situations: if your critical parameter is in the "well-defined" space, then you
may use it as a biomarker. If, on the other hand, this parameter is in the
degenerate space, it cannot be used since its value is not stable. The burden
of proof is of course on the presenter. One can support it by
- small RSEs on the parameter of interest, if you can get them;
- no correlation with other parameters (either in bootstrap samples, or in the
history of SAEM iterations, or by investigation the variance-covariance matrix
of the parameter estimates);
- starting the model run with perturbed values of this parameter to show that
the final estimate does not depend on the initial values;
- etc.
Alternative is to try to make the entire model stable by fixing some parameters
at the biologically plausible values: if the model with fixed parameters is
still flexible enough to describe the entire range of available data, one can
use this model until some experimental results provide the data (and the need)
to free and estimate those fixed parameters.
Thanks
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Quoted reply history
On 9/7/2010 2:42 AM, Steingötter Andreas wrote:
Hello Nick, Leonid, Dieter
As a beginner in NONMEM society, I am becoming very curious in your current
discussion. In a related situation I may need some helpful comments and already
excuse myself if this question has been answered many times before.
THE SITUATION: We have tissue (let's say tumor tissue) that has some anatomical
structure known by histology. So we know roughly how many blood vessels (to get
an idea on blood flow/perfusion), how many vital tissue (to get an idea where
the blood can distribute or perfuse into) and how many dead tissue (where only
blood diffusion can take place) is present. We inject a substance (i.v. bolus)
and macroscopically follow its kinetics through this tissue, i.e. have a
concentration curve of this tissue. At the same time we can also measure the
kinetics of other (neighboring) healthy tissues to generate additional
concentration curves. All these curves exhibit bi- or multi-exponential
behavior.
First PROBLEM: We only observe on the macroscopic scale and therefore we have a
mixture of tissue kinetics for each concentration curve. However, we are able
to create a model that perfectly describes the concentration curves of all
tissues as Dieter has done. This model is very likely to be over-parameterized.
In a SECOND STEP we treat this tumor tissue and see some changes in tumor
structure. But don't have clue how these changes in structure relate to changes
in function, e.g. what rate constant, volume flow or distribution volume is
most sensitive to such a change in blood vessels. For later purpose and to omit
the need for histology the aim is to identify this sensitive parameter and use
it as some kind of biomarker.
NOW THE QUESTION: How to best proceed to (numerically) find this most sensitive
parameter in the model? Do we start from the model that best describes the
concentration curves and go backwards again. Do we pick a first potential
parameter and reduce the model until this parameter is robust (shows no
correlation) and do the same again for other possible candidates? Do we then
end up with one model for each parameter of interest (which does not make sense
to me)?
To my understanding, for a given (rich) data set there can only be a compromise
between model fit and robustness of parameter estimation and finally someone
has to decide what that is. This compromise then needs to be tested and
validated again and again by generating or including new data.
BEGINNER's QUESTION: If we show that we have done the testing and tweaking with
regard to what we (pretend to) know from physiology/biology/histology and are
aware of (and describe) the uncertainty in parameter estimates for the
selected, probably over parameterized model, would expert reviewers of your
caliber still ask for more model simplification?
Sorry for being so elaborate and many thanks for comments and critics of every
description.
Andreas
Andreas Steingötter, PhD
Division for Gastroenterology and Hepatology
Department of Internal Medicine
University Hospital Zurich
Von: [email protected] [mailto:[email protected]] Im
Auftrag von Nick Holford
Gesendet: Dienstag, 7. September 2010 04:19
An: nmusers
Betreff: Re: [NMusers] How serious are negative eigenvalues?
Dieter,
You ask:
My question: can we trust this fit?
The answer depends on why you are doing the modelling.
If your goal is to describe the time course of concentrations then the overall
ability of the model to describe what you saw depends on the totality of the
model and its parameters. The model may be overparameterized but it may still
do what you want it to do i.e. describe (and predict) the time course of
concentrations in each compartment. If you are satisfied with the VPC showing
that simulations from the model appropriately describe the observed
concentrations then I think the answer to your question is yes.
On the other hand if the goal is to estimate the size of one or more critical
parameters then you will need to pay attention to how well these parameters are
estimated. As Leonid has pointed out it seems that at least some of the model
parameters are not well identified. This may be unimportant if the parameters
you want to describe are robustly estimated.
For example, if you had a simple PK model with samples mainly taken at steady
state with few observations during absorption then you may get a good estimate
of clearance but a rather poor estimate of KA. You cannot simply remove a
parameter such as KA (you have to describe the sparse absorption somehow) but
it will have little impact on the clearance estimate. Thus the model can be
trusted for the purpose of estimating clearance but not absorption rate.
Nick
On 7/09/2010 12:11 a.m., Dieter Menne wrote:
Dear Nmusers,
we have very rich data from MRI concentration measurements, with 11
compartments and multiple compartments observed. The model is fit via SAEM
(nburn=2000), and followed by an IMPMAP as in the described in the 7.1.2
manual. OMEGA is band with pair-wise block correlations in the following
style:
$OMEGA BLOCK(2)
.02 ;CL
0.01 0.06 ; VC
$OMEGA BLOCK(2)
5.4 ; QMVP
0.001 0.05 ;VMVP
$OMEGA BLOCK(2)
0.06 ; QTVP
0.001 0.25 ;VTPV
$EST PRINT=1 METHOD=SAEM INTERACTION NBURN=2000 NITER=200 CTYPE=2 NSIG=2
FILE=SAEM.EXT
$EST METHOD=IMPMAP EONLY = 1 INTERACTION ISAMPLE=1000 NITER=5 FILE=IMP.EXT
$COV PRINT=E UNCONDITIONAL
Fits and CWRES diagnostics are perfect, and VPC checks are good.
However, we have negative eigenvalues (the following example has been edited
by removing digits)
ETAPval = 0.2 0.2 0.3 0.04 0.8 0.95 0.003 0.1 0.6 0.4 0.9 0.1 0.5 0.4 0.2
0.8 0.3 0.3 0.4 0.01 0.8
ETAshr% = 13. 0.4 38 20 23 33 46 30 18 41 54 22 2. 26. 49. 12. 0.07 24. 18.
35. 2.5
EPSshr% = 7.5 8.1
Number of Negative Eigenvalues in Matrix= 7
Most negative value= -65339.
Most positive value= 88796185.9
Forcing positive definiteness
Root mean square deviation of matrix from original= 1.37E-003
My question: can we trust this fit?
Dieter Menne
Menne Biomed/University Hospital of Zürich
--
Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology
University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
email: [email protected]
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford