Parameter uncertainty

11 messages 10 people Latest: Feb 16, 2017

Parameter uncertainty

From: Fanny Gallais Date: February 15, 2017 technical
Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais

RE: Parameter uncertainty

From: DJ Eleveld-Ufkes Date: February 15, 2017 technical
Hi Fanny, Likelihood profiles are very useful to asses parameter uncertainty. I am sure you find a tutorial somewhere how they work. A number of software packages automate the process quite a bit. They are usually much more computationally efficient than bootstrap. Warm regards, Douglas Eleveld
Quoted reply history
From: [email protected] [mailto:[email protected]] On Behalf Of Fanny Gallais Sent: woensdag 15 februari 2017 11:55 To: [email protected] Subject: [NMusers] Parameter uncertainty Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais ________________________________

RE: Parameter uncertainty

From: Max Taubert Date: February 15, 2017 technical
Dear Fanny, I would use either bootstrapping or likelihood profiling, both of them are implemented in PsN ('bootstrap' and 'llp'). Kind regards Max Taubert ________________________________ Von: [email protected] [[email protected]]" im Auftrag von "Fanny Gallais [[email protected]] Gesendet: Mittwoch, 15. Februar 2017 11:55 An: [email protected] Betreff: [NMusers] Parameter uncertainty Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais

Re: Parameter uncertainty

From: William Denney Date: February 15, 2017 technical
Hi Fanny, It is often good practice to fit parameters that must be positive on the log scale (by exponentiating them). That will ensure that when sampling from a normal distribution (and then exponentiating the sample) you will have a positive value. LLP was suggested, but it won't assess correlation between your parameters which is often important when running simulations. Bootstrap is another good alternative as has already been suggested. Thanks, Bill
Quoted reply history
> On Feb 15, 2017, at 5:55 AM, Fanny Gallais <[email protected]> wrote: > > Dear NM users, > > I would like to perform a simulation (on R) incorporating parameter > uncertainty. For now I'm working on a simple PK model. Parameters were > estimated with NONMEM. I'm trying to figure out what is the best way to > assess parameter uncertainty. I've read about using the standard errors > reported by NONMEM and assume a normal distribution. The main problem is this > can lead to negative values. Another approach would be a more computational > non-parametric method like bootstrap. Do you know other methods to assess > parameter uncertainty? > > > Best regards > > F. Gallais > > > > >

RE: Parameter uncertainty

From: Pieter Colin Date: February 15, 2017 technical
Hi Fanny, As I understand it, you’re looking for ways to produce predictions according to your model taking into account parameter uncertainty. We’ve recently published on the importance of parameter uncertainty when considering probability of target attainment for antibiotic dosing regimens. (Colin et al. J Antimicrob Chemother (2016) 71 (9): 2502-2508) The online supplement to this paper holds an R-script which you can use to simulate (and calculate PTA, if relevant) taking into account parameter uncertainty. For this, the script uses the variance-covariance matrix that is produced by the $COV step in NONMEM. Of course other techniques which generate a var-cov matrix could be used as input for the script as well. Kind regards, Pieter Colin
Quoted reply history
From: [email protected] [mailto:[email protected]] On Behalf Of Fanny Gallais Sent: woensdag 15 februari 2017 11:55 To: [email protected] Subject: [NMusers] Parameter uncertainty Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais

RE: Parameter uncertainty

From: Martin Bergstrand Date: February 15, 2017 technical
Dear Fanny and Bill, The sampling importance resampling (SIR) approach [1] to characterize the parameter uncertainty address the aspects pointed out Bill. In my opinion this is currently in general the most widely applicable and accurate method to characterize parameter uncertainty for NLMEM (bootstrap is likely approximately as good for large datasets and balanced designs). The method is implemented in PsN [2] and ready to use together with NONMEM. [1] Dosne A-G, Bergstrand M, Harling K, Karlsson MO. Improving the estimation of parameter uncertainty distributions in nonlinear mixed effects models using sampling importance resampling. J Pharmacokinet Pharmacodyn. 2016 Oct 11. http://link.springer.com/article/10.1007/s10928-016-9487-8 [2] SIR user guide, PsN 4.6.0: http://psn.sourceforge.net/pdfdocs/sir_userguide.pdf Best regards, Martin Bergstrand, Ph.D. Senior Consultant Pharmetheus AB +46(0)709 994 396 [email protected] www.pharmetheus.com +46(0)18 513 328 U-A Science Park, Dag Hammarskjölds v. 52b 752 37 Uppsala, Sweden *This communication is confidential and is only intended for the use of the individual or entity to which it is directed. It may contain information that is privileged and exempt from disclosure under applicable law. If you are not the intended recipient please notify us immediately. Please do not copy it or disclose its contents to any other person.* *From:* [email protected] [mailto:[email protected]] *On Behalf Of *William Denney *Sent:* Wednesday, February 15, 2017 1:01 PM *To:* Fanny Gallais <[email protected]> *Cc:* [email protected] *Subject:* Re: [NMusers] Parameter uncertainty Hi Fanny, It is often good practice to fit parameters that must be positive on the log scale (by exponentiating them). That will ensure that when sampling from a normal distribution (and then exponentiating the sample) you will have a positive value. LLP was suggested, but it won't assess correlation between your parameters which is often important when running simulations. Bootstrap is another good alternative as has already been suggested. Thanks, Bill
Quoted reply history
On Feb 15, 2017, at 5:55 AM, Fanny Gallais <[email protected]> wrote: Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais

Re: FW: Parameter uncertainty

From: Leonid Gibiansky Date: February 15, 2017 technical
One of the tools available for simulations is Metrum R package metrumrg install.packages("metrumrg", repos=" http://R-Forge.R-project.org";) Example of applications can be found here: http://www.page-meeting.org/page/page2006/P2006III_11.pdf Since the time it was written (2005-2006), Nonmem enhanced the simulations options, so now you can simulate from the model-estimated uncertainty directly from Nonmem. The R package could be useful if you do it from the bootstrap results. Leonid > *From:*[email protected] > [mailto:[email protected]] *On Behalf Of *Fanny Gallais > *Sent:* Wednesday, February 15, 2017 5:55 AM > *To:* [email protected] > *Subject:* [NMusers] Parameter uncertainty > > Dear NM users, > > I would like to perform a simulation (on R) incorporating parameter > uncertainty. For now I'm working on a simple PK model. Parameters were > estimated with NONMEM. I'm trying to figure out what is the best way to > assess parameter uncertainty. I've read about using the standard errors > reported by NONMEM and assume a normal distribution. The main problem is > this can lead to negative values. Another approach would be a more > computational non-parametric method like bootstrap. Do you know other > methods to assess parameter uncertainty? > > Best regards > > F. Gallais > > *NOTICE: *The information contained in this electronic mail message is > intended only for the personal and confidential use of the designated > recipient(s) named above. This message may be an attorney-client > communication, may be protected by the work product doctrine, and may be > subject to a protective order. As such, this message is privileged and > confidential. If the reader of this message is not the intended > recipient or an agent responsible for delivering it to the intended > recipient, you are hereby notified that you have received this message > in error and that any review, dissemination, distribution, or copying of > this message is strictly prohibited. If you have received this > communication in error, please notify us immediately by telephone and > e-mail and destroy any and all copies of this message in your possession > (whether hard copies or electronically stored copies). Thank you. > > Personal data may be transferred to the United States of America and, if > this occurs, it is possible that US governmental authorities may access > such personal data. > > buSp9xeMeKEbrUze

RE: Parameter uncertainty

From: Jason Williams Date: February 15, 2017 technical
Dear Fanny, Another useful tool you may want to try is using the mrgsolve package available in R, developed by Kyle Baron at Metrum Research Group. I have found mrgsolve to be very efficient for PKPD simulation and sensitivity analysis in R. There is an example of incorporating parameter uncertainty (from $COV step in NONMEM) in Section 9 of the example on Probability of Technical Success (link below). https://github.com/mrgsolve/examples/blob/master/PrTS/pts.pdf Best regards, Jason
Quoted reply history
From: [email protected] [mailto:[email protected]] On Behalf Of Fanny Gallais Sent: Wednesday, February 15, 2017 2:55 AM To: [email protected] Subject: [NMusers] Parameter uncertainty Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais

Re: Parameter uncertainty

From: Fanny Gallais Date: February 16, 2017 technical
Thank you all for your responses. It is going to be very useful for my work. Best regards, F.G.
Quoted reply history
2017-02-15 17:35 GMT+01:00 Williams, Jason <[email protected]>: > Dear Fanny, > > > > Another useful tool you may want to try is using the mrgsolve package > available in R, developed by Kyle Baron at Metrum Research Group. I have > found mrgsolve to be very efficient for PKPD simulation and sensitivity > analysis in R. There is an example of incorporating parameter uncertainty > (from $COV step in NONMEM) in Section 9 of the example on Probability of > Technical Success (link below). > > > > https://github.com/mrgsolve/examples/blob/master/PrTS/pts.pdf > > > > Best regards, > > > Jason > > > > *From:* [email protected] [mailto:[email protected]] > *On Behalf Of *Fanny Gallais > *Sent:* Wednesday, February 15, 2017 2:55 AM > *To:* [email protected] > *Subject:* [NMusers] Parameter uncertainty > > > > Dear NM users, > > > > I would like to perform a simulation (on R) incorporating parameter > uncertainty. For now I'm working on a simple PK model. Parameters were > estimated with NONMEM. I'm trying to figure out what is the best way to > assess parameter uncertainty. I've read about using the standard errors > reported by NONMEM and assume a normal distribution. The main problem is > this can lead to negative values. Another approach would be a more > computational non-parametric method like bootstrap. Do you know other > methods to assess parameter uncertainty? > > > > > > Best regards > > > > F. Gallais > > > > > > >

Re: Parameter uncertainty

From: Marc Gastonguay Date: February 16, 2017 technical
Dear Fanny, One additional method to obtain the parameter uncertainty, which I don't believe was mentioned, is Bayesian estimation using Markov-Chain Monte Carlo (MCMC) simulation. This method provides a full joint posterior distribution (e.g. uncertainty distribution) of the parameters and any predicted quantities, and is really the gold standard for this type of goal. It is possible to implement this method in NONMEM (with some limitations on the prior distributions), or you could use BUGS or Stan with associated PK model libraries. You can also extract the samples from the posterior distribution and simulate using the methods already described in this thread. Marc
Quoted reply history
On Thu, Feb 16, 2017 at 6:01 AM, Fanny Gallais <[email protected]> wrote: > Thank you all for your responses. It is going to be very useful for my > work. > > Best regards, > > F.G. > > 2017-02-15 17:35 GMT+01:00 Williams, Jason <[email protected]>: > >> Dear Fanny, >> >> >> >> Another useful tool you may want to try is using the mrgsolve package >> available in R, developed by Kyle Baron at Metrum Research Group. I have >> found mrgsolve to be very efficient for PKPD simulation and sensitivity >> analysis in R. There is an example of incorporating parameter uncertainty >> (from $COV step in NONMEM) in Section 9 of the example on Probability of >> Technical Success (link below). >> >> >> >> https://github.com/mrgsolve/examples/blob/master/PrTS/pts.pdf >> >> >> >> Best regards, >> >> >> Jason >> >> >> >> *From:* [email protected] [mailto:[email protected]] >> *On Behalf Of *Fanny Gallais >> *Sent:* Wednesday, February 15, 2017 2:55 AM >> *To:* [email protected] >> *Subject:* [NMusers] Parameter uncertainty >> >> >> >> Dear NM users, >> >> >> >> I would like to perform a simulation (on R) incorporating parameter >> uncertainty. For now I'm working on a simple PK model. Parameters were >> estimated with NONMEM. I'm trying to figure out what is the best way to >> assess parameter uncertainty. I've read about using the standard errors >> reported by NONMEM and assume a normal distribution. The main problem is >> this can lead to negative values. Another approach would be a more >> computational non-parametric method like bootstrap. Do you know other >> methods to assess parameter uncertainty? >> >> >> >> >> >> Best regards >> >> >> >> F. Gallais >> >> >> >> >> >> >> > > -- Marc R. Gastonguay, Ph.D. <[email protected]> CEO Metrum Research Group LLC http://metrumrg.com 2 Tunxis Rd., Ste 112, Tariffville, CT 06081 USA Tel: +1.860.735.7043 ext. 101, Mobile: +1.860.670.0744, Fax: +1.860.760.6014

RE: Parameter uncertainty

From: Jacob Leander Date: February 16, 2017 technical
Hi Fanny, Marc I was thinking in the same direction as Marc. If you use MCMC (BAYES method in NONMEM) the algorithm will provide you with samples from the posterior density (posterior = likelihood * prior). From these samples you can then investigate different statistics, for example variance of your parameters. Be caution about convergence of the algorithm, since these algorithms are not guaranteed to sample uncorrelated samples. On the same topic, are there any good comparisons out there comparing the standard covariance matrix approach, bootstrap, profiling and MCMC? /Jacob
Quoted reply history
From: [email protected] [mailto:[email protected]] On Behalf Of Marc Gastonguay Sent: den 16 februari 2017 13:23 To: Fanny Gallais <[email protected]> Cc: Williams, Jason <[email protected]>; [email protected] Subject: Re: [NMusers] Parameter uncertainty Dear Fanny, One additional method to obtain the parameter uncertainty, which I don't believe was mentioned, is Bayesian estimation using Markov-Chain Monte Carlo (MCMC) simulation. This method provides a full joint posterior distribution (e.g. uncertainty distribution) of the parameters and any predicted quantities, and is really the gold standard for this type of goal. It is possible to implement this method in NONMEM (with some limitations on the prior distributions), or you could use BUGS or Stan with associated PK model libraries. You can also extract the samples from the posterior distribution and simulate using the methods already described in this thread. Marc On Thu, Feb 16, 2017 at 6:01 AM, Fanny Gallais <[email protected]<mailto:[email protected]>> wrote: Thank you all for your responses. It is going to be very useful for my work. Best regards, F.G. 2017-02-15 17:35 GMT+01:00 Williams, Jason <[email protected]<mailto:[email protected]>>: Dear Fanny, Another useful tool you may want to try is using the mrgsolve package available in R, developed by Kyle Baron at Metrum Research Group. I have found mrgsolve to be very efficient for PKPD simulation and sensitivity analysis in R. There is an example of incorporating parameter uncertainty (from $COV step in NONMEM) in Section 9 of the example on Probability of Technical Success (link below). https://github.com/mrgsolve/examples/blob/master/PrTS/pts.pdf Best regards, Jason From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Fanny Gallais Sent: Wednesday, February 15, 2017 2:55 AM To: [email protected]<mailto:[email protected]> Subject: [NMusers] Parameter uncertainty Dear NM users, I would like to perform a simulation (on R) incorporating parameter uncertainty. For now I'm working on a simple PK model. Parameters were estimated with NONMEM. I'm trying to figure out what is the best way to assess parameter uncertainty. I've read about using the standard errors reported by NONMEM and assume a normal distribution. The main problem is this can lead to negative values. Another approach would be a more computational non-parametric method like bootstrap. Do you know other methods to assess parameter uncertainty? Best regards F. Gallais -- Marc R. Gastonguay, Ph.D.<mailto:[email protected]> CEO Metrum Research Group http://metrumrg.com 2 Tunxis Rd., Ste 112, Tariffville, CT 06081 USA Tel: +1.860.735.7043 ext. 101, Mobile: +1.860.670.0744, Fax: +1.860.760.6014 ________________________________ Confidentiality Notice: This message is private and may contain confidential and proprietary information. If you have received this message in error, please notify us and remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorized use or disclosure of the contents of this message is not permitted and may be unlawful.