RE: ETAs & SIGMA in external validation

From: [email protected] [mailto:[email protected]] On Behalf Of Faelens, Ruben (Belgium) Sent: maandag 9 april 2018 23:52 To: Tingjie Guo; [email protected] Subject: RE: [NMusers] ETAs & SIGMA in external validation Hi Tingjie, A lot of great tips and explanations already. Just wanted to add my two cents. POSTHOC will estimate the most likely ETA for each individual, taking into account the known population parameters THETA and OMEGA. “Most likely” means: 1. An individual parameter as close as possible to the typical value (i.e. ETA=0). The likelihood of an ETA is evaluated through the probability density function of a normal distribution with mean 0 and variance OMEGA. 2. A model prediction as close as possible to the observed values. The likelihood of a model prediction is evaluated through the probability density function of the residual error model. We find the most likely ETA by using maximum likelihood estimation (I do not know the exact algorithm, but I use Nelder-Mead in my own software and that produces the same results as nonmem). You have two questions: 1. Can I constrict the ETA search space so only realistic ETA’s are found? You can, but that would change your original model, and would require re-estimating THETA and OMEGA. For some parameters (e.g. disease progression, or LOG(BASELINE) ), an absolute inter-individual variability on a parameter may make sense. You may want to re-evaluate (as suggested previously) whether this is valid for all parameters. In other words: whether the parameter IIV is truly symmetric normal distributed. In any case, posthoc estimations are linked to the original model. If close-to-zero parameter values are unlikely to appear in the training dataset, then OMEGA should be small, and therefore negative values of a parameter will probably not be estimated anyway (part 1 of our maximum likelihood estimation explained above). And if the model does not make any sense with negative parameter values, the model predictions will be very far off from the observed values as well (part 2 of our maximum likelihood estimation). I suggest you re-evaluate the ETA distributions of your original model, and consider using a lognormal IIV instead. You could also explore graphically the input data for subjects with negative ETA values. Possibly the observed input data can only be explained through negative parameter values? @Jakob: Could you explain how “The solution you initially implemented will bias the parameter distribution severely, since only values greater than or equal to the typical parameter value is allowed.” ? In case of an IIV of e.g. 20% CV, (1+ETA) would require 5 standard deviations on ETA before it becomes negative. 1. Which error model should I use? Should I only use the assay error? Residual error comes from many sources. Assay error is only one of these. Others include model misspecification, dosing errors, true dose deviations (e.g. use of generics, or inaccuracies in preparing an infusion), bad recording of sample times, etc. Unless there is a good reason to assume your new data was not subjected to the same errors as the training dataset, you should keep the same residual error model. I myself am still struggling with this question: “Should we again sample residual error when we simulate from EBE estimates? Or should we estimate individual parameter uncertainty from the OFIM and use only that?” Best regards, Ruben Faelens Scientist at SGS Exprimo PhD Student at KULeuven From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Tingjie Guo Sent: vrijdag 6 april 2018 18:32 To: [email protected]<mailto:[email protected]> Subject: [NMusers] ETAs & SIGMA in external validation Dear NMusers, I have two questions regarding the statistical model when performing external validation. I have a dataset and would like to validate a published model with POSTHOC method i.e. $EST METHOD=0 POSTHOC MAXEVAL=0. 1. The model added etas in proportional way, i.e. Para = THETA * (1+ETA) and this made the posthoc estimation fail due to the negative individual parameter estimate in some subjects. I constrained it to be positive by adding ABS function i.e. Para = THETA * ABS(1+ETA), and the estimation can be successfully running. I was wondering if there is better workaround? 2. OMEGA value influences individual ETAs in POSTHOC estimation. Should we assign $SIGMA with model value or lab (where external data was determined) assay error value? If we use model value, it's understandable that $SIGMA contains unexplained variability and thus it is a part of the model. However, I may also understand it as that model value contains the unexplained variability for original data (in which the model was created) but not for external data. I'm a little confused about it. Can someone help me out? I would appreciate any response! Many thanks in advance! Your sincerely, Tingjie Guo Information in this email and any attachments is confidential and intended solely for the use of the individual(s) to whom it is addressed or otherwise directed. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the Company. Finally, the recipient should check this email and any attachments for the presence of viruses. The Company accepts no liability for any damage caused by any virus transmitted by this email. All SGS services are rendered in accordance with the applicable SGS conditions of service available on request and accessible at http://www.sgs.com/en/Terms-and-Conditions.aspx ________________________________