Dear all,
thank you for the useful input.
I agree with Jeroen about the fact that "those model parts that describe most of variance in the most plausible manner should be introduced first".
In fact, I can think of a couple of situations in my not so long experience, in which the inclusion of a very significant covariate was necessary to correctly identify some of the other components of the model.
We had a study where some patients were sampled in two occasions, once while given only the drug under test, and another time with co-administration of a known inducer. If the effect of this inducer was not taken into account, the model was not able to separate BSV and BOV for CL. In another case, if a similar covariate effect was not included, BOV in bioavailability was not found significant, while it greatly improved the model, if incorporated after accounting for the covariate.
In other words, I guess there's no rule that will work 100% of the times, but my feeling is that, even in the worst case scenario, the ETAs (both BOV and BSV) are very easy to remove from a model, since the corresponding OMEGA will tend to shrink as they become less significant. Also, the inclusion of the ETAs and the inspection of their plots against time and other covariates might help to identify significant covariate or time-dependent effects, as long as the shrinkage is not too large. This was suggested to me by Martin Bergstrand in a private message.
Finally, I also agree with Bill about the fact that not always we will reach the same "best" model independently of the modelling strategy employed.. Obvioulsy re-testing some assumptions along the way may be a more robust approach, but there's probably no complete guarantee...
So probably Oscar is right, you should brush your teeth again and again... But again, probably modelling cannot be compared to only a simple breakfast, it is much rather a multi-course meal... ;)
Regards,
Paolo
Quoted reply history
On 24/11/2010 00:12, Denney, William S. wrote:
> Hi Jeroen,
>
> Jumping in a bit later, I agree generally with what has been said so far, but I do
> disagree with one point. I think that the models we work with tend to have local minima
> that cause us to find different "best models" depending on the path taken to
> get there.
>
> And, I brush after breakfast to preserve the taste of the meal.
>
> Thanks,
>
> Bill
>
> On Nov 23, 2010, at 4:49 PM, "Elassaiss - Schaap, J.
> (Jeroen)"<[email protected]> wrote:
>
> > Hi Paolo,
> >
> > It is a bit late to chime in but I can't resist... Great discussion point! I am
> > of the opinion that if we develop models robustly, it in the end should not
> > matter. If we would introduce a structural bias by neglecting BOV early on, we
> > should be able to see a reflection of that in a diagnostic plot after
> > introduction of BOV. And that in turn should lead to evaluation of other
> > structural models. But this obviously depends on close scrutiny of diagnostics
> > and frequent back-tracing.
> >
> > Perhaps the question could be restated as: which method is more efficient? -
> > retaining the original answer.
> >
> > It may even be generalized by stating that those model parts that describe most
> > of variance in the most plausible manner should be introduced first. This
> > should prevent bias that complicates evaluation of more detailed parts because
> > of nonlinearity issues as you described.
> >
> > Such a rule could be applied to any model and result in e.g. BSV on baseline be
> > added early on for a PK-PD problem, body weight for general PK, BOV for
> > multi-occasion/rich sampling problems, to name a few.
> >
> > Last but not least, I skip breakfast completely ;-).
> >
> > Best regards,
> > Jeroen
> >
> > Modeling& Simulation Expert
> > Pharmacokinetics, Pharmacodynamics& Pharmacometrics (P3) - DMPK
> > MSD
> > PO Box 20 - AP1112
> > 5340 BH Oss
> > The Netherlands
> > [email protected]
> > T: +31 (0)412 66 9320
> > M: +31 (0)6 46 101 283
> > F: +31 (0)412 66 2506
> > www.msd.com
> >
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]] On
> > Behalf Of Paolo Denti
> > Sent: Wednesday, 17 November, 2010 17:23
> > To: Elodie Plan
> > Cc: 'nmusers'
> > Subject: Re: [NMusers] Zähneputzen VOR oder NACH dem Frühstück? What comes
> > first? BSV, BOV, or covariates?
> >
> > Thank you Elodie,
> > the reference you mention also states that the covariates were tested only on parameters
> > for which BOV and BSV were significant. This is generally the approach I use, so that I
> > can test whether the mentioned variabilities are indeed explained with the inclusion of
> > covariates. I wonder if somebody can think of any exceptions to this "rule"?
> >
> > Also, both Oscar della Pasqua and Coen Van Hasselt pointed to me this PAGE
> > poster (unfortunately presented in a literally burning hot poster session in
> > Berlin):
> > http://www.page-meeting.org/default.asp?abstract=1887
> > which seems to stress that disregarding BOV might lead to model
> > misspecification.
> >
> > I also got a reply from Alwin Huitema, who told me that his experience with
> > modelling in HIV is that ignoring IOV early in the modelling process might
> > guide to wrong models.
> >
> > Any supporters of an alternative approach or shall I just assume that I was
> > doing the same as everybody else?
> >
> > Who would brush teeth before breakfast anyway? ;) Another, safer, option is
> > suggested by Oscar:
> >
> > > Paolo,
> > >
> > > By the way, hygiene rules do suggest you brush your teeth before and after
> > > breakfast.
> > > I don't want to infer that this is the same for modelling but I can
> > > say that you can recognise the individual ingredients in your
> > > breakfast if your taste butts are clean:)
> > >
> > > Oscar
> >
> > Ciao,
> > Paolo
> >
> > On 16/11/2010 22:15, Elodie Plan wrote:
> >
> > > Dear Paolo,
> > >
> > > Thanks for this interesting NMusers thread.
> > >
> > > I think the order you are describing really makes sense in theory, for
> > > the reasons you describe, but in brief because it seems covariates
> > > should be incorporated on a model already fully developed structurally
> > > and statistically, so this includes IOV. Moreover, the covariates will
> > > increase the predictive performance (and the understanding) of the
> > > model, by being introduced on structural parameters, but also possibly
> > > directly on IIV and IOV.
> > >
> > > I also wanted to verify that this was what was done in practice, there
> > > were
> > > 6 entries when searching for "occasion AND covariate AND NONMEM" on
> > > PubMed, I can recommend the following where the decrease in
> > > variability magnitude following the covariate model building is nicely
> > > discussed: Sandström M, Lindman H, Nygren P, Johansson M, Bergh J,
> > > Karlsson MO. Population analysis of the pharmacokinetics and the
> > > haematological toxicity of the fluorouracil-epirubicin-cyclophosphamide regimen
> > > in breast cancer patients.
> > > Cancer Chemother Pharmacol. 2006 Aug;58(2):143-56.
> > >
> > > Best regards,
> > > Elodie
> > >
> > > PS: IOV or breakfast, I like it first :)
> > >
> > > Elodie L. Plan, PharmD, MSc, PhD student
> > > ********************************************
> > > Uppsala Pharmacometrics Research Group Department of Pharmaceutical
> > > Biosciences P.O. Box 591, SE-751 24 Uppsala, SWEDEN Mob +46 76-242
> > > 1256, Skype "ppeloo"
> > >
> > > -----Original Message-----
> > > From:[email protected]
> > > [mailto:[email protected]] On Behalf Of Paolo Denti
> > > Sent: Tuesday, November 16, 2010 10:10 AM
> > > To: nmusers
> > > Subject: [NMusers] Zähneputzen VOR oder NACH dem Frühstück? What comes
> > > first? BSV, BOV, or covariates?
> > >
> > > Dear all,
> > > don't be discouraged by the subject, this is indeed NMUsers and not
> > > German 101, and this post is about pharmacometrics, please read on...
> > > ;)
> > >
> > > The subject of the message comes from when I was studying German, and
> > > from an exercise in our book with lots of colourful pictures. The
> > > point of the exercise was only to teach us how to say "tooth
> > > brushing", "have breakfast", "before" and "after", but instead it
> > > sprouted a lively discussion in the class about what comes first and
> > > last in everybody's morning routine... So I thought it would be an
> > > appropriate title for this post, which is a survey/question about what
> > > modelling approach people use/recommend for model development.
> > >
> > > Just to contextualize a bit, here at UCT we mainly study HIV and TB
> > > drugs, which are dosed repeatedly (once or twice per day) and administered
> > > orally.
> > > We often have data available on more than one sampling occasion, and
> > > many times these occasions are virtually
> > > equivalent: no changes in co-treatment or other covariates, just a
> > > mere repetition of the experiment on a different day. Confirming what
> > > Mats recently pointed out in a post about the use of BOV, our
> > > experience is that, especially in the absorption phase, the
> > > contribution of BOV is dominant, and cannot be ignored. The absorption
> > > is often subject to random delays and factors that are mostly
> > > occasion-specific and not measurable/available in the dataset.
> > >
> > > Therefore, when I start modelling new data, I normally proceed as follows:
> > > 1. I initially assume every occasion as a separate profile, either
> > > using dummy IDs (and pretending it's different subjects) or coding all
> > > variability as BOV. I believe this allows the maximum flexibility to
> > > test the structural model, and I find that, if I don't proceed like
> > > this, I may run into troubles detecting the correct structural model.
> > > In this early stage of model development, I mostly use individual
> > > plots, and try to see if my prediction profile is flexible enough to run
> > > through the points.
> > >
> > > 2. Then I try to see if some of the variability is subject-specific
> > > (normally V and CL) and can be better explained either by only BSV or
> > > both BSV and BOV. I use the OFV to guide this process, but if the BOV
> > > is much larger than BSV, and physiology supports the hypothesis that
> > > the parameter be occasion-specific, I tend to disregard BSV.
> > >
> > > 3. Once I believe I got my structural model right, and organized the
> > > hierarchy of random variability in a decent way, I start incorporating
> > > the covariates. If they turn out to be significant, I see that BOV and
> > > BSV decrease, and sometimes become superfluous in the model and can be removed.
> > >
> > > I know other modellers would recommend first introducing BSV and/or
> > > covariates, before considering BOV and I would be interested in
> > > knowing people's opinion about this. Each method probably has its pros
> > > and cons, and I would really value your input about this topic. What
> > > are the advantages and disadvantages of the different approaches?
> > >
> > > Since I favour the modus operandi I just explained, I give my reasons,
> > > and look forward to some comments. My opinion (but I am obviously
> > > biased) is that it does not hurt to include BOV first, since it is
> > > easy to remove from the model if the same variability is explained by
> > > covariates, and likely, if this is the case, BOV will decrease in size.
> > > On the other hand, disregarding BOV might prevent the identification
> > > of the correct structural model. I am thinking, for example, about a
> > > comparison between 2-cmpt vs 1-cmpt when the absorption is subject to
> > > substantial random delays. If BOV is not considered, this is
> > > equivalent to pooling the data from all occasions, with the potential
> > > result of having a cloud of points without much structure... And also,
> > > as a general rule, I would allow a parameter to move with an ETA,
> > > before I try to explain its changes with a covariate effect. In this
> > > way I can also test better if the covariate is explaining some of this
> > > variability.
> > >
> > > Ok, I've been once again way too lengthy, apologies. Any comments/thoughts?
> > > In other words, do you first brush your teeth or have breakfast?
> > > Please join the survey! ;)
> > >
> > > Greetings from Cape Town,
> > > Paolo
> > >
> > > PS Ich putze die Zähne immer NACH dem Frühstück... I can't enjoy
> > > coffee with that minty toothpaste after-taste... :)
> > >
> > > --
> > > ------------------------------------------------
> > > Paolo Denti, PhD
> > > Post-Doctoral Fellow
> > > Division of Clinical Pharmacology
> > > Department of Medicine
> > > University of Cape Town
> > >
> > > K45 Old Main Building
> > > Groote Schuur Hospital
> > > Observatory, Cape Town
> > > 7925 South Africa
> > > phone: +27 21 404 7719
> > > fax: +27 21 448 1989
> > > email:[email protected]
> > > ------------------------------------------------
> > >
> > > ###
> > > UNIVERSITY OF CAPE TOWN
> > >
> > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer
> > > published on our website at
> > > http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable
> > > from +27
> > > 21 650 9111. This e-mail is intended only for the person(s) to whom it
> > > is addressed. If the e-mail has reached you in error, please notify the author.
> > > If you are not the intended recipient of the e-mail you may not use,
> > > disclose, copy, redirect or print the content. If this e-mail is not
> > > related to the business of UCT it is sent by the sender in the
> > > sender's individual capacity.
> > >
> > > ###
> >
> > --
> > ------------------------------------------------
> > Paolo Denti, PhD
> > Post-Doctoral Fellow
> > Division of Clinical Pharmacology
> > Department of Medicine
> > University of Cape Town
> >
> > K45 Old Main Building
> > Groote Schuur Hospital
> > Observatory, Cape Town
> > 7925 South Africa
> > phone: +27 21 404 7719
> > fax: +27 21 448 1989
> > email:[email protected]
> > ------------------------------------------------
> >
> > ###
> > UNIVERSITY OF CAPE TOWN
> >
> > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published
> > on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or
> > obtainable from +27 21 650 9111. This e-mail is intended only for the person(s)
> > to whom it is addressed. If the e-mail has reached you in error, please notify
> > the author. If you are not the intended recipient of the e-mail you may not
> > use, disclose, copy, redirect or print the content. If this e-mail is not
> > related to the business of UCT it is sent by the sender in the sender's
> > individual capacity.
> >
> > ###
> >
> > This message and any attachments are solely for the intended recipient. If you
> > are not the intended recipient, disclosure, copying, use or distribution of the
> > information included in this message is prohibited --- Please immediately and
> > permanently delete.
--
------------------------------------------------
Paolo Denti, PhD
Post-Doctoral Fellow
Division of Clinical Pharmacology
Department of Medicine
University of Cape Town
K45 Old Main Building
Groote Schuur Hospital
Observatory, Cape Town
7925 South Africa
phone: +27 21 404 7719
fax: +27 21 448 1989
email: [email protected]
------------------------------------------------
###
UNIVERSITY OF CAPE TOWN
This e-mail is subject to the UCT ICT policies and e-mail disclaimer published
on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or
obtainable from +27 21 650 9111. This e-mail is intended only for the person(s)
to whom it is addressed. If the e-mail has reached you in error, please notify
the author. If you are not the intended recipient of the e-mail you may not
use, disclose, copy, redirect or print the content. If this e-mail is not
related to the business of UCT it is sent by the sender in the sender's
individual capacity.
###