Reporting and handling values below the limit of quantification

On 7/11/2013 4:33 a.m., Paolo Denti wrote: > Dear all, > I know I am opening a bit of a can of worms here, and one that has been > opened before, but please bear with me.. > > We are trying to make our case with our analytical laboratory to > convince them to release to us (pharmacometrics) the values below the > limit of quantification (BLQ), which they normally define as the level > below which they can't guarantee 20% CV on the measurement. > > So far, they have been quite reluctant, because they say that this would > go against their SOPs, quality assurance policies, some FDA and EMA > guidelines, and what not. However, after months of insisting, it seems > like they may finally be open for discussion and asked us to present as > much supporting evidence and experience from other labs as possible. > > Our main argument is that censoring BLQ values may be a reasonable > policy when the data needs to be used for other purposes or by > clinicians, but for us modelers it is a terrible waste of information, > because we have tools to properly deal with the additional level of > uncertainty, > > My first question to the group is then the following - Nick, I > explicitly count on you for this one... :) > 1. Can you suggest any literature/guidelines/references in support of > our cause? > a. Any literature clearly advocating for/supporting the release of the > BLQ values for pharmacometric modelling. > b. Any official guidelines providing/justifying an exception to the > standard practice of censoring when the data is handled with modelling > c. Any personal experience with your lab or the regulatory authority > about this topic > > So far, I've found some previous threads here on NMUsers and the > conclusion section in this paper: > Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an > arbitrary quantification limit on structural model misspecification. J. > Pharmacokinet. Pharmacodyn. 35: 101–16, 2008. > > The second question is about how to handle these values if we manage to > get them (fingers crossed). > The released data will have some actual values below the assay > validation limit (that we can call "low precision"), and some that will > be NA, because sometimes the mass-spec will not be able to identify a > peak in the elution profile. > > 2. What error structure would you recommend to handle a dataset > including uncensored BLQ values? > a. Should one fix the additive component of the error to a fraction of > the LLOQ (say 50%)? And if so, for all samples, even the ones above > LLOQ, or only the BLQ ones? > b. How would you handle the NAs? Would you impute 0? Impute the lowest > value reported? Half of it? > c. If you have a series of NAs to impute, would you retain only the > first one and exclude the following, or would include them all? Would > you have the proportional component of the error apply also to the > imputed NAs or not? > > Any input and help is greatly appreciated! > > Greetings from Cape Town, > Paolo > > -- > ------------------------------------------------ > Paolo Denti, PhD > Pharmacometrics Group > Division of Clinical Pharmacology > Department of Medicine > University of Cape Town > > K45 Old Main Building > Groote Schuur Hospital > Observatory, Cape Town > 7925 South Africa > phone: +27 21 404 7719 > fax: +27 21 448 1989 > email: [email protected] > ------------------------------------------------ > > ________________________________ > UNIVERSITY OF CAPE TOWN > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 email: [email protected] http://holford.fmhs.auckland.ac.nz/ Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics and Pharmacodynamics. 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and adults. J Pharm Sci. 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html Holford NHG. Clinical pharmacology = disease progression + drug action. British Journal of Clinical Pharmacology. 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract

Van: [email protected] [mailto:[email protected]] Namens Paolo Denti Verzonden: woensdag 6 november 2013 16:33 Aan: NMusers Onderwerp: [NMusers] Reporting and handling values below the limit of quantification Dear all, I know I am opening a bit of a can of worms here, and one that has been opened before, but please bear with me.. We are trying to make our case with our analytical laboratory to convince them to release to us (pharmacometrics) the values below the limit of quantification (BLQ), which they normally define as the level below which they can't guarantee 20% CV on the measurement. So far, they have been quite reluctant, because they say that this would go against their SOPs, quality assurance policies, some FDA and EMA guidelines, and what not. However, after months of insisting, it seems like they may finally be open for discussion and asked us to present as much supporting evidence and experience from other labs as possible. Our main argument is that censoring BLQ values may be a reasonable policy when the data needs to be used for other purposes or by clinicians, but for us modelers it is a terrible waste of information, because we have tools to properly deal with the additional level of uncertainty, My first question to the group is then the following - Nick, I explicitly count on you for this one... :) 1. Can you suggest any literature/guidelines/references in support of our cause? a. Any literature clearly advocating for/supporting the release of the BLQ values for pharmacometric modelling. b. Any official guidelines providing/justifying an exception to the standard practice of censoring when the data is handled with modelling c. Any personal experience with your lab or the regulatory authority about this topic So far, I've found some previous threads here on NMUsers and the conclusion section in this paper: Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an arbitrary quantification limit on structural model misspecification. J. Pharmacokinet. Pharmacodyn. 35: 101-16, 2008. The second question is about how to handle these values if we manage to get them (fingers crossed). The released data will have some actual values below the assay validation limit (that we can call "low precision"), and some that will be NA, because sometimes the mass-spec will not be able to identify a peak in the elution profile. 2. What error structure would you recommend to handle a dataset including uncensored BLQ values? a. Should one fix the additive component of the error to a fraction of the LLOQ (say 50%)? And if so, for all samples, even the ones above LLOQ, or only the BLQ ones? b. How would you handle the NAs? Would you impute 0? Impute the lowest value reported? Half of it? c. If you have a series of NAs to impute, would you retain only the first one and exclude the following, or would include them all? Would you have the proportional component of the error apply also to the imputed NAs or not? Any input and help is greatly appreciated! Greetings from Cape Town, Paolo -- ------------------------------------------------ Paolo Denti, PhD Pharmacometrics Group Division of Clinical Pharmacology Department of Medicine University of Cape Town K45 Old Main Building Groote Schuur Hospital Observatory, Cape Town 7925 South Africa phone: +27 21 404 7719 fax: +27 21 448 1989 email: [email protected] ------------------------------------------------ ________________________________ UNIVERSITY OF CAPE TOWN This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. ***************************DISCLAIMER**************************** De informatie in dit e-mail bericht is uitsluitend bestemd voor de geadresseerde. Verstrekking aan en gebruik door anderen is niet toegestaan. Door de elektronische verzending van het bericht kunnen er geen rechten worden ontleend aan de informatie.

> On 2013/11/11 12:17, [email protected] wrote: > > > Hi Paolo, > > > > Considering 2A: > > > > The residual error model always should already account for a possible larger > > residual error at lower concentrations. I think a combined proportional and > > additive error model willfor this. If an additive component can't be estimated, > > my gut feeling tells me fixing the additive component to 1/2BLQ is reasonable. > > > > Cheers, > > Rob On 2013/11/09 06:55, Nick Holford wrote: > Paulo, > > I wish you luck in trying to do this. I also spend some time trying to persuade the people in your lab to do intelligent things with their measurements when I was in Cape Town. > > I suggest you might try looking at this: > http://holford.fmhs.auckland.ac.nz/docs/censored-observations-with-nonmem.pdf > > I discuss the FDA Guidance that is usually used by the chemical analysts to support their deliberate attempts to make our life difficult. > > Unfortunately this is largely an issue of belief not science. It is essentially impossible to win religious battles with wisdom. The usual strategy to win a religious war is with guns and bombs. I don't recommend that. But perhaps you might try drugs -- e.g. your excellent South African wine. > > In Auckland I was able to persuade one LC-MS chemical analyst to see the light and he reported his measurements honestly (including some negative concentration measurements). A complex PK model was published based on these truthful observations (Patel et al. 2011). > > Once you have honest observations I think it is much easier to decide how to model the residual error. The additive error component can then be a realistic description of assay background noise. > > Best wishes, > > Nick > > Patel K, Choy SS, Hicks KO, Melink TJ, Holford NH, Wilson WR. A combined pharmacokinetic model for the hypoxia-targeted prodrug PR-104A in humans, dogs, rats and mice predicts species differences in clearance and toxicity. Cancer Chemother Pharmacol. 2011;67(5):1145-55. > > On 7/11/2013 4:33 a.m., Paolo Denti wrote: > > > Dear all, > > I know I am opening a bit of a can of worms here, and one that has been > > opened before, but please bear with me.. > > > > We are trying to make our case with our analytical laboratory to > > convince them to release to us (pharmacometrics) the values below the > > limit of quantification (BLQ), which they normally define as the level > > below which they can't guarantee 20% CV on the measurement. > > > > So far, they have been quite reluctant, because they say that this would > > go against their SOPs, quality assurance policies, some FDA and EMA > > guidelines, and what not. However, after months of insisting, it seems > > like they may finally be open for discussion and asked us to present as > > much supporting evidence and experience from other labs as possible. > > > > Our main argument is that censoring BLQ values may be a reasonable > > policy when the data needs to be used for other purposes or by > > clinicians, but for us modelers it is a terrible waste of information, > > because we have tools to properly deal with the additional level of > > uncertainty, > > > > My first question to the group is then the following - Nick, I > > explicitly count on you for this one... :) > > 1. Can you suggest any literature/guidelines/references in support of > > our cause? > > a. Any literature clearly advocating for/supporting the release of the > > BLQ values for pharmacometric modelling. > > b. Any official guidelines providing/justifying an exception to the > > standard practice of censoring when the data is handled with modelling > > c. Any personal experience with your lab or the regulatory authority > > about this topic > > > > So far, I've found some previous threads here on NMUsers and the > > conclusion section in this paper: > > Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an > > arbitrary quantification limit on structural model misspecification. J. > > Pharmacokinet. Pharmacodyn. 35: 101–16, 2008. > > > > The second question is about how to handle these values if we manage to > > get them (fingers crossed). > > The released data will have some actual values below the assay > > validation limit (that we can call "low precision"), and some that will > > be NA, because sometimes the mass-spec will not be able to identify a > > peak in the elution profile. > > > > 2. What error structure would you recommend to handle a dataset > > including uncensored BLQ values? > > a. Should one fix the additive component of the error to a fraction of > > the LLOQ (say 50%)? And if so, for all samples, even the ones above > > LLOQ, or only the BLQ ones? > > b. How would you handle the NAs? Would you impute 0? Impute the lowest > > value reported? Half of it? > > c. If you have a series of NAs to impute, would you retain only the > > first one and exclude the following, or would include them all? Would > > you have the proportional component of the error apply also to the > > imputed NAs or not? > > > > Any input and help is greatly appreciated! > > > > Greetings from Cape Town, > > Paolo > > > > -- > > ------------------------------------------------ > > Paolo Denti, PhD > > Pharmacometrics Group > > Division of Clinical Pharmacology > > Department of Medicine > > University of Cape Town > > > > K45 Old Main Building > > Groote Schuur Hospital > > Observatory, Cape Town > > 7925 South Africa > > phone: +27 21 404 7719 > > fax: +27 21 448 1989 > > email: [email protected] > > ------------------------------------------------ > > > > ________________________________ > > UNIVERSITY OF CAPE TOWN > > > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > email:[email protected] > http://holford.fmhs.auckland.ac.nz/ > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > and Pharmacodynamics. > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > adults. J Pharm Sci. > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > Holford NHG. Clinical pharmacology = disease progression + drug action. British > Journal of Clinical Pharmacology. > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract -- ------------------------------------------------ Paolo Denti, PhD Pharmacometrics Group Division of Clinical Pharmacology Department of Medicine University of Cape Town K45 Old Main Building Groote Schuur Hospital Observatory, Cape Town 7925 South Africa phone: +27 21 404 7719 fax: +27 21 448 1989 email: [email protected]

On 12/11/2013 2:29 a.m., Paolo Denti wrote: > Dear Nick and Rob (and all), > thanks for your answers. > > Alwin Huitema pointed out to me this work done with Ron Keizer: > http://www.page-meeting.org/default.asp?abstract=1722 > > Nick, I will try the wine for extra help to let people "see the light", but it seems like I may be lucky this time. :) They agreed in principle to release the data for "academic work", while for work scrutinised by the regulatory authorities they need to stick to the guidelines. Hopefully, if more and more publications report the use of BLQ concentrations, the message will spread that releasing these concentrations and handling them properly is more reasonable than censoring. > > I have seen the presentation on Nick's site, especially Stuart Beal's suggestion on the margin of the slides advocating fixing the additive component to LLOQ/2 (or LLOQ/5). Do you think the suggestion applies only when the imputation is used, or is it a good idea in general when using BLQ concentrations? > > Rob, I was also thinking of using the combined error and fixing the additive component to LLOQ/2 (see the slides in Nick's link with the Stuart Beal's suggestion), if necessary. The question is when is it necessary to fix vs. estimate? If the OFV suggests an estimate for the additive component much less than LLOQ/2, would you just trust that? Or would you use LLOQ/2 (or LLOQ/5) as a lower bound for the estimate of the additive error no matter what the OFV says? > > Thanks again, > Paolo > > > On 2013/11/11 12:17, [email protected] wrote: > > > > > Hi Paolo, > > > > > > Considering 2A: > > > > > > The residual error model always should already account for a possible larger > > > residual error at lower concentrations. I think a combined proportional and > > > additive error model willfor this. If an additive component can't be estimated, > > > my gut feeling tells me fixing the additive component to 1/2BLQ is reasonable. > > > > > > Cheers, > > > Rob > > On 2013/11/09 06:55, Nick Holford wrote: > > > Paulo, > > > > I wish you luck in trying to do this. I also spend some time trying to persuade the people in your lab to do intelligent things with their measurements when I was in Cape Town. > > > > I suggest you might try looking at this: > > http://holford.fmhs.auckland.ac.nz/docs/censored-observations-with-nonmem.pdf > > > > I discuss the FDA Guidance that is usually used by the chemical analysts to support their deliberate attempts to make our life difficult. > > > > Unfortunately this is largely an issue of belief not science. It is essentially impossible to win religious battles with wisdom. The usual strategy to win a religious war is with guns and bombs. I don't recommend that. But perhaps you might try drugs -- e.g. your excellent South African wine. > > > > In Auckland I was able to persuade one LC-MS chemical analyst to see the light and he reported his measurements honestly (including some negative concentration measurements). A complex PK model was published based on these truthful observations (Patel et al. 2011). > > > > Once you have honest observations I think it is much easier to decide how to model the residual error. The additive error component can then be a realistic description of assay background noise. > > > > Best wishes, > > > > Nick > > > > Patel K, Choy SS, Hicks KO, Melink TJ, Holford NH, Wilson WR. A combined pharmacokinetic model for the hypoxia-targeted prodrug PR-104A in humans, dogs, rats and mice predicts species differences in clearance and toxicity. Cancer Chemother Pharmacol. 2011;67(5):1145-55. > > > > On 7/11/2013 4:33 a.m., Paolo Denti wrote: > > > > > Dear all, > > > I know I am opening a bit of a can of worms here, and one that has been > > > opened before, but please bear with me.. > > > > > > We are trying to make our case with our analytical laboratory to > > > convince them to release to us (pharmacometrics) the values below the > > > limit of quantification (BLQ), which they normally define as the level > > > below which they can't guarantee 20% CV on the measurement. > > > > > > So far, they have been quite reluctant, because they say that this would > > > > > > go against their SOPs, quality assurance policies, some FDA and EMA > > > guidelines, and what not. However, after months of insisting, it seems > > > like they may finally be open for discussion and asked us to present as > > > much supporting evidence and experience from other labs as possible. > > > > > > Our main argument is that censoring BLQ values may be a reasonable > > > policy when the data needs to be used for other purposes or by > > > clinicians, but for us modelers it is a terrible waste of information, > > > because we have tools to properly deal with the additional level of > > > uncertainty, > > > > > > My first question to the group is then the following - Nick, I > > > explicitly count on you for this one... :) > > > 1. Can you suggest any literature/guidelines/references in support of > > > our cause? > > > a. Any literature clearly advocating for/supporting the release of the > > > BLQ values for pharmacometric modelling. > > > b. Any official guidelines providing/justifying an exception to the > > > standard practice of censoring when the data is handled with modelling > > > c. Any personal experience with your lab or the regulatory authority > > > about this topic > > > > > > So far, I've found some previous threads here on NMUsers and the > > > conclusion section in this paper: > > > Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an > > > arbitrary quantification limit on structural model misspecification. J. > > > Pharmacokinet. Pharmacodyn. 35: 101–16, 2008. > > > > > > The second question is about how to handle these values if we manage to > > > get them (fingers crossed). > > > The released data will have some actual values below the assay > > > validation limit (that we can call "low precision"), and some that will > > > be NA, because sometimes the mass-spec will not be able to identify a > > > peak in the elution profile. > > > > > > 2. What error structure would you recommend to handle a dataset > > > including uncensored BLQ values? > > > a. Should one fix the additive component of the error to a fraction of > > > the LLOQ (say 50%)? And if so, for all samples, even the ones above > > > LLOQ, or only the BLQ ones? > > > b. How would you handle the NAs? Would you impute 0? Impute the lowest > > > value reported? Half of it? > > > c. If you have a series of NAs to impute, would you retain only the > > > first one and exclude the following, or would include them all? Would > > > you have the proportional component of the error apply also to the > > > imputed NAs or not? > > > > > > Any input and help is greatly appreciated! > > > > > > Greetings from Cape Town, > > > Paolo > > > > > > -- > > > ------------------------------------------------ > > > Paolo Denti, PhD > > > Pharmacometrics Group > > > Division of Clinical Pharmacology > > > Department of Medicine > > > University of Cape Town > > > > > > K45 Old Main Building > > > Groote Schuur Hospital > > > Observatory, Cape Town > > > 7925 South Africa > > > phone: +27 21 404 7719 > > > fax: +27 21 448 1989 > > > email: [email protected] > > > ------------------------------------------------ > > > > > > ________________________________ > > > UNIVERSITY OF CAPE TOWN > > > > > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. > > > > -- > > Nick Holford, Professor Clinical Pharmacology > > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > > email:[email protected] > > http://holford.fmhs.auckland.ac.nz/ > > > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > > and Pharmacodynamics. > > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > > adults. J Pharm Sci. > > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > > Holford NHG. Clinical pharmacology = disease progression + drug action. British > > Journal of Clinical Pharmacology. > > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract > > -- > ------------------------------------------------ > Paolo Denti, PhD > Pharmacometrics Group > Division of Clinical Pharmacology > Department of Medicine > University of Cape Town > > K45 Old Main Building > Groote Schuur Hospital > Observatory, Cape Town > 7925 South Africa > phone: +27 21 404 7719 > fax: +27 21 448 1989 > email:[email protected] > ------------------------------------------------ -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 email: [email protected] http://holford.fmhs.auckland.ac.nz/ Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics and Pharmacodynamics. 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and adults. J Pharm Sci. 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html Holford NHG. Clinical pharmacology = disease progression + drug action. British Journal of Clinical Pharmacology. 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract

On 2013/11/11 21:42, Nick Holford wrote: > Paulo, > > The note associated with my Slide 7 is not intended to be a current recommendation but is intended to show the history of ideas. Stuart Beal examined several methods in his 2001 paper (see Slide 8) and concluded that the likelihood based methods (M2, M3, M4) were better than the imputation methods (M5-M7: e.g. replace BLQ with LLOQ/2). Beal's paper examined using imputed values and does not mention fixing the additive error variance in the description of these methods (perhaps I missed this detail somewhere in this complex paper). > > I don't know of any study that has examined the performance of fixing the additive error (eg. var(eps(2)) fixed to .25*QL**2) in combination with using an imputed value. It is not based on the stronger statistical theory that is at the heart of the likelihood based methods which also allow the estimation of the unknown additive residual error variance so I would not use it unless someone has evaluated its statistical properties like the other methods. > > Given that the likelihood methods are so easy to use I really don't know why you are considering this method. My own preference is to use M3 if the chemical analysts cannot be persuaded to be honest by reporting the actual measured value. M2 (implemented using YLO in NM-TRAN) is not as robust and M4 takes longer but does not seem to have any clear advantage in simulation based evaluations. > > Best wishes, > > Nick > > On 12/11/2013 2:29 a.m., Paolo Denti wrote: > > > Dear Nick and Rob (and all), > > thanks for your answers. > > > > Alwin Huitema pointed out to me this work done with Ron Keizer: > > http://www.page-meeting.org/default.asp?abstract=1722 > > > > Nick, I will try the wine for extra help to let people "see the light", but it seems like I may be lucky this time. :) They agreed in principle to release the data for "academic work", while for work scrutinised by the regulatory authorities they need to stick to the guidelines. Hopefully, if more and more publications report the use of BLQ concentrations, the message will spread that releasing these concentrations and handling them properly is more reasonable than censoring. > > > > I have seen the presentation on Nick's site, especially Stuart Beal's suggestion on the margin of the slides advocating fixing the additive component to LLOQ/2 (or LLOQ/5). Do you think the suggestion applies only when the imputation is used, or is it a good idea in general when using BLQ concentrations? > > > > Rob, I was also thinking of using the combined error and fixing the additive component to LLOQ/2 (see the slides in Nick's link with the Stuart Beal's suggestion), if necessary. The question is when is it necessary to fix vs. estimate? If the OFV suggests an estimate for the additive component much less than LLOQ/2, would you just trust that? Or would you use LLOQ/2 (or LLOQ/5) as a lower bound for the estimate of the additive error no matter what the OFV says? > > > > Thanks again, > > Paolo > > > > > On 2013/11/11 12:17, [email protected] wrote: > > > > > > > Hi Paolo, > > > > > > > > Considering 2A: > > > > > > > > The residual error model always should already account for a possible larger > > > > residual error at lower concentrations. I think a combined proportional and > > > > additive error model willfor this. If an additive component can't be estimated, > > > > my gut feeling tells me fixing the additive component to 1/2BLQ is reasonable. > > > > > > > > Cheers, > > > > Rob > > > > On 2013/11/09 06:55, Nick Holford wrote: > > > > > Paulo, > > > > > > I wish you luck in trying to do this. I also spend some time trying to persuade the people in your lab to do intelligent things with their measurements when I was in Cape Town. > > > > > > I suggest you might try looking at this: > > > http://holford.fmhs.auckland.ac.nz/docs/censored-observations-with-nonmem.pdf > > > > > > I discuss the FDA Guidance that is usually used by the chemical analysts to support their deliberate attempts to make our life difficult. > > > > > > Unfortunately this is largely an issue of belief not science. It is essentially impossible to win religious battles with wisdom. The usual strategy to win a religious war is with guns and bombs. I don't recommend that. But perhaps you might try drugs -- e.g. your excellent South African wine. > > > > > > In Auckland I was able to persuade one LC-MS chemical analyst to see the light and he reported his measurements honestly (including some negative concentration measurements). A complex PK model was published based on these truthful observations (Patel et al. 2011). > > > > > > Once you have honest observations I think it is much easier to decide how to model the residual error. The additive error component can then be a realistic description of assay background noise. > > > > > > Best wishes, > > > > > > Nick > > > > > > Patel K, Choy SS, Hicks KO, Melink TJ, Holford NH, Wilson WR. A combined pharmacokinetic model for the hypoxia-targeted prodrug PR-104A in humans, dogs, rats and mice predicts species differences in clearance and toxicity. Cancer Chemother Pharmacol. 2011;67(5):1145-55. > > > > > > On 7/11/2013 4:33 a.m., Paolo Denti wrote: > > > > > > > Dear all, > > > > > > > > I know I am opening a bit of a can of worms here, and one that has been > > > > > > > > opened before, but please bear with me.. > > > > > > > > We are trying to make our case with our analytical laboratory to > > > > convince them to release to us (pharmacometrics) the values below the > > > > limit of quantification (BLQ), which they normally define as the level > > > > below which they can't guarantee 20% CV on the measurement. > > > > > > > > So far, they have been quite reluctant, because they say that this would > > > > > > > > go against their SOPs, quality assurance policies, some FDA and EMA > > > > guidelines, and what not. However, after months of insisting, it seems > > > > > > > > like they may finally be open for discussion and asked us to present as > > > > > > > > much supporting evidence and experience from other labs as possible. > > > > > > > > Our main argument is that censoring BLQ values may be a reasonable > > > > policy when the data needs to be used for other purposes or by > > > > clinicians, but for us modelers it is a terrible waste of information, > > > > because we have tools to properly deal with the additional level of > > > > uncertainty, > > > > > > > > My first question to the group is then the following - Nick, I > > > > explicitly count on you for this one... :) > > > > 1. Can you suggest any literature/guidelines/references in support of > > > > our cause? > > > > a. Any literature clearly advocating for/supporting the release of the > > > > BLQ values for pharmacometric modelling. > > > > b. Any official guidelines providing/justifying an exception to the > > > > standard practice of censoring when the data is handled with modelling > > > > c. Any personal experience with your lab or the regulatory authority > > > > about this topic > > > > > > > > So far, I've found some previous threads here on NMUsers and the > > > > conclusion section in this paper: > > > > Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an > > > > > > > > arbitrary quantification limit on structural model misspecification. J. > > > > > > > > Pharmacokinet. Pharmacodyn. 35: 101–16, 2008. > > > > > > > > The second question is about how to handle these values if we manage to > > > > > > > > get them (fingers crossed). > > > > The released data will have some actual values below the assay > > > > > > > > validation limit (that we can call "low precision"), and some that will > > > > > > > > be NA, because sometimes the mass-spec will not be able to identify a > > > > peak in the elution profile. > > > > > > > > 2. What error structure would you recommend to handle a dataset > > > > including uncensored BLQ values? > > > > a. Should one fix the additive component of the error to a fraction of > > > > the LLOQ (say 50%)? And if so, for all samples, even the ones above > > > > LLOQ, or only the BLQ ones? > > > > b. How would you handle the NAs? Would you impute 0? Impute the lowest > > > > value reported? Half of it? > > > > c. If you have a series of NAs to impute, would you retain only the > > > > first one and exclude the following, or would include them all? Would > > > > you have the proportional component of the error apply also to the > > > > imputed NAs or not? > > > > > > > > Any input and help is greatly appreciated! > > > > > > > > Greetings from Cape Town, > > > > Paolo > > > > > > > > -- > > > > ------------------------------------------------ > > > > Paolo Denti, PhD > > > > Pharmacometrics Group > > > > Division of Clinical Pharmacology > > > > Department of Medicine > > > > University of Cape Town > > > > > > > > K45 Old Main Building > > > > Groote Schuur Hospital > > > > Observatory, Cape Town > > > > 7925 South Africa > > > > phone: +27 21 404 7719 > > > > fax: +27 21 448 1989 > > > > email: [email protected] > > > > ------------------------------------------------ > > > > > > > > ________________________________ > > > > UNIVERSITY OF CAPE TOWN > > > > > > > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. > > > > > > -- > > > Nick Holford, Professor Clinical Pharmacology > > > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > > > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > > > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > > > email:[email protected] > > > http://holford.fmhs.auckland.ac.nz/ > > > > > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > > > and Pharmacodynamics. > > > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > > > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > > > adults. J Pharm Sci. > > > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > > > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > > > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > > > Holford NHG. Clinical pharmacology = disease progression + drug action. British > > > Journal of Clinical Pharmacology. > > > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract > > > > -- > > ------------------------------------------------ > > Paolo Denti, PhD > > Pharmacometrics Group > > Division of Clinical Pharmacology > > Department of Medicine > > University of Cape Town > > > > K45 Old Main Building > > Groote Schuur Hospital > > Observatory, Cape Town > > 7925 South Africa > > phone: +27 21 404 7719 > > fax: +27 21 448 1989 > > email:[email protected] > > ------------------------------------------------ > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > email:[email protected] > http://holford.fmhs.auckland.ac.nz/ > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > and Pharmacodynamics. > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > adults. J Pharm Sci. > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > Holford NHG. Clinical pharmacology = disease progression + drug action. British > Journal of Clinical Pharmacology. > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract -- ------------------------------------------------ Paolo Denti, PhD Pharmacometrics Group Division of Clinical Pharmacology Department of Medicine University of Cape Town K45 Old Main Building Groote Schuur Hospital Observatory, Cape Town 7925 South Africa phone: +27 21 404 7719 fax: +27 21 448 1989 email:[email protected]

On 12/11/2013 9:08 p.m., Paolo Denti wrote: > Hi Nick, > I think there is a misunderstanding, my question was not clear. > > I am asking for the general opinion about fixing the additive component of the error when the raw BLQ data has been released and NOT censored. As a general idea, I would always estimate parameters rather than fixing them, but I was thinking that in this case it may be more conservative to impose a minimum value for the additive component, e.g. 20% or 50% of the LLOQ. > > I guess I will play a bit with simulations and see how much this matters. > > Thank you, > Paolo > > On 2013/11/11 21:42, Nick Holford wrote: > > > Paulo, > > > > The note associated with my Slide 7 is not intended to be a current recommendation but is intended to show the history of ideas. Stuart Beal examined several methods in his 2001 paper (see Slide 8) and concluded that the likelihood based methods (M2, M3, M4) were better than the imputation methods (M5-M7: e.g. replace BLQ with LLOQ/2). Beal's paper examined using imputed values and does not mention fixing the additive error variance in the description of these methods (perhaps I missed this detail somewhere in this complex paper). > > > > I don't know of any study that has examined the performance of fixing the additive error (eg. var(eps(2)) fixed to .25*QL**2) in combination with using an imputed value. It is not based on the stronger statistical theory that is at the heart of the likelihood based methods which also allow the estimation of the unknown additive residual error variance so I would not use it unless someone has evaluated its statistical properties like the other methods. > > > > Given that the likelihood methods are so easy to use I really don't know why you are considering this method. My own preference is to use M3 if the chemical analysts cannot be persuaded to be honest by reporting the actual measured value. M2 (implemented using YLO in NM-TRAN) is not as robust and M4 takes longer but does not seem to have any clear advantage in simulation based evaluations. > > > > Best wishes, > > > > Nick > > > > On 12/11/2013 2:29 a.m., Paolo Denti wrote: > > > > > Dear Nick and Rob (and all), > > > thanks for your answers. > > > > > > Alwin Huitema pointed out to me this work done with Ron Keizer: > > > http://www.page-meeting.org/default.asp?abstract=1722 > > > > > > Nick, I will try the wine for extra help to let people "see the light", but it seems like I may be lucky this time. :) They agreed in principle to release the data for "academic work", while for work scrutinised by the regulatory authorities they need to stick to the guidelines. Hopefully, if more and more publications report the use of BLQ concentrations, the message will spread that releasing these concentrations and handling them properly is more reasonable than censoring. > > > > > > I have seen the presentation on Nick's site, especially Stuart Beal's suggestion on the margin of the slides advocating fixing the additive component to LLOQ/2 (or LLOQ/5). Do you think the suggestion applies only when the imputation is used, or is it a good idea in general when using BLQ concentrations? > > > > > > Rob, I was also thinking of using the combined error and fixing the additive component to LLOQ/2 (see the slides in Nick's link with the Stuart Beal's suggestion), if necessary. The question is when is it necessary to fix vs. estimate? If the OFV suggests an estimate for the additive component much less than LLOQ/2, would you just trust that? Or would you use LLOQ/2 (or LLOQ/5) as a lower bound for the estimate of the additive error no matter what the OFV says? > > > > > > Thanks again, > > > Paolo > > > > > > > On 2013/11/11 12:17, [email protected] wrote: > > > > > > > > > Hi Paolo, > > > > > > > > > > Considering 2A: > > > > > > > > > > The residual error model always should already account for a possible larger > > > > > residual error at lower concentrations. I think a combined proportional and > > > > > additive error model willfor this. If an additive component can't be estimated, > > > > > my gut feeling tells me fixing the additive component to 1/2BLQ is reasonable. > > > > > > > > > > Cheers, > > > > > Rob > > > > > > On 2013/11/09 06:55, Nick Holford wrote: > > > > > > > Paulo, > > > > > > > > I wish you luck in trying to do this. I also spend some time trying to persuade the people in your lab to do intelligent things with their measurements when I was in Cape Town. > > > > > > > > I suggest you might try looking at this: > > > > http://holford.fmhs.auckland.ac.nz/docs/censored-observations-with-nonmem.pdf > > > > > > > > I discuss the FDA Guidance that is usually used by the chemical analysts to support their deliberate attempts to make our life difficult. > > > > > > > > Unfortunately this is largely an issue of belief not science. It is essentially impossible to win religious battles with wisdom. The usual strategy to win a religious war is with guns and bombs. I don't recommend that. But perhaps you might try drugs -- e.g. your excellent South African wine. > > > > > > > > In Auckland I was able to persuade one LC-MS chemical analyst to see the light and he reported his measurements honestly (including some negative concentration measurements). A complex PK model was published based on these truthful observations (Patel et al. 2011). > > > > > > > > Once you have honest observations I think it is much easier to decide how to model the residual error. The additive error component can then be a realistic description of assay background noise. > > > > > > > > Best wishes, > > > > > > > > Nick > > > > > > > > Patel K, Choy SS, Hicks KO, Melink TJ, Holford NH, Wilson WR. A combined pharmacokinetic model for the hypoxia-targeted prodrug PR-104A in humans, dogs, rats and mice predicts species differences in clearance and toxicity. Cancer Chemother Pharmacol. 2011;67(5):1145-55. > > > > > > > > On 7/11/2013 4:33 a.m., Paolo Denti wrote: > > > > > > > > > Dear all, > > > > > > > > > > I know I am opening a bit of a can of worms here, and one that has been > > > > > > > > > > opened before, but please bear with me.. > > > > > > > > > > We are trying to make our case with our analytical laboratory to > > > > > convince them to release to us (pharmacometrics) the values below the > > > > > > > > > > limit of quantification (BLQ), which they normally define as the level > > > > > > > > > > below which they can't guarantee 20% CV on the measurement. > > > > > > > > > > So far, they have been quite reluctant, because they say that this would > > > > > > > > > > go against their SOPs, quality assurance policies, some FDA and EMA > > > > > > > > > > guidelines, and what not. However, after months of insisting, it seems like they may finally be open for discussion and asked us to present as > > > > > > > > > > much supporting evidence and experience from other labs as possible. > > > > > > > > > > Our main argument is that censoring BLQ values may be a reasonable > > > > > policy when the data needs to be used for other purposes or by > > > > > > > > > > clinicians, but for us modelers it is a terrible waste of information, > > > > > > > > > > because we have tools to properly deal with the additional level of > > > > > uncertainty, > > > > > > > > > > My first question to the group is then the following - Nick, I > > > > > explicitly count on you for this one... :) > > > > > 1. Can you suggest any literature/guidelines/references in support of > > > > > our cause? > > > > > > > > > > a. Any literature clearly advocating for/supporting the release of the > > > > > > > > > > BLQ values for pharmacometric modelling. > > > > > b. Any official guidelines providing/justifying an exception to the > > > > > > > > > > standard practice of censoring when the data is handled with modelling > > > > > > > > > > c. Any personal experience with your lab or the regulatory authority > > > > > about this topic > > > > > > > > > > So far, I've found some previous threads here on NMUsers and the > > > > > conclusion section in this paper: > > > > > Byon W, Fletcher C V, Brundage RC. Impact of censoring data below an > > > > > > > > > > arbitrary quantification limit on structural model misspecification. J. > > > > > > > > > > Pharmacokinet. Pharmacodyn. 35: 101–16, 2008. > > > > > > > > > > The second question is about how to handle these values if we manage to > > > > > > > > > > get them (fingers crossed). > > > > > The released data will have some actual values below the assay > > > > > > > > > > validation limit (that we can call "low precision"), and some that will > > > > > > > > > > be NA, because sometimes the mass-spec will not be able to identify a > > > > > peak in the elution profile. > > > > > > > > > > 2. What error structure would you recommend to handle a dataset > > > > > including uncensored BLQ values? > > > > > > > > > > a. Should one fix the additive component of the error to a fraction of > > > > > > > > > > the LLOQ (say 50%)? And if so, for all samples, even the ones above > > > > > LLOQ, or only the BLQ ones? > > > > > > > > > > b. How would you handle the NAs? Would you impute 0? Impute the lowest > > > > > > > > > > value reported? Half of it? > > > > > c. If you have a series of NAs to impute, would you retain only the > > > > > first one and exclude the following, or would include them all? Would > > > > > you have the proportional component of the error apply also to the > > > > > imputed NAs or not? > > > > > > > > > > Any input and help is greatly appreciated! > > > > > > > > > > Greetings from Cape Town, > > > > > Paolo > > > > > > > > > > -- > > > > > ------------------------------------------------ > > > > > Paolo Denti, PhD > > > > > Pharmacometrics Group > > > > > Division of Clinical Pharmacology > > > > > Department of Medicine > > > > > University of Cape Town > > > > > > > > > > K45 Old Main Building > > > > > Groote Schuur Hospital > > > > > Observatory, Cape Town > > > > > 7925 South Africa > > > > > phone: +27 21 404 7719 > > > > > fax: +27 21 448 1989 > > > > > email: [email protected] > > > > > ------------------------------------------------ > > > > > > > > > > ________________________________ > > > > > UNIVERSITY OF CAPE TOWN > > > > > > > > > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. > > > > > > > > -- > > > > Nick Holford, Professor Clinical Pharmacology > > > > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > > > > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > > > > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > > > > email:[email protected] > > > > http://holford.fmhs.auckland.ac.nz/ > > > > > > > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > > > > and Pharmacodynamics. > > > > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > > > > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > > > > adults. J Pharm Sci. > > > > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > > > > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > > > > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > > > > Holford NHG. Clinical pharmacology = disease progression + drug action. British > > > > Journal of Clinical Pharmacology. > > > > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract > > > > > > -- > > > ------------------------------------------------ > > > Paolo Denti, PhD > > > Pharmacometrics Group > > > Division of Clinical Pharmacology > > > Department of Medicine > > > University of Cape Town > > > > > > K45 Old Main Building > > > Groote Schuur Hospital > > > Observatory, Cape Town > > > 7925 South Africa > > > phone: +27 21 404 7719 > > > fax: +27 21 448 1989 > > > email:[email protected] > > > ------------------------------------------------ > > > > -- > > Nick Holford, Professor Clinical Pharmacology > > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand > > office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 > > email:[email protected] > > http://holford.fmhs.auckland.ac.nz/ > > > > Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics > > and Pharmacodynamics. > > 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 > > Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and > > adults. J Pharm Sci. > > 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract > > Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. > > 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html > > Holford NHG. Clinical pharmacology = disease progression + drug action. British > > Journal of Clinical Pharmacology. > > 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract > > -- > ------------------------------------------------ > Paolo Denti, PhD > Pharmacometrics Group > Division of Clinical Pharmacology > Department of Medicine > University of Cape Town > > K45 Old Main Building > Groote Schuur Hospital > Observatory, Cape Town > 7925 South Africa > phone: +27 21 404 7719 > fax: +27 21 448 1989 > email:[email protected] > ------------------------------------------------ -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 email: [email protected] http://holford.fmhs.auckland.ac.nz/ Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics and Pharmacodynamics. 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and adults. J Pharm Sci. 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html Holford NHG. Clinical pharmacology = disease progression + drug action. British Journal of Clinical Pharmacology. 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract