Reporting of interim models in Pop PK Reports

From: "Bonate, Peter" Peter.Bonate@genzyme.com Subject: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 10:00:23 -0400 I was just reading the new CHMP guidance on Pop PK Reports. The guidance recommends including the final structural and covariate model from NONMEM (control stream and listing) as an appendix. It makes no mention of interim models. So this got me thinking (which can be a bad thing in itself) that in my current pop pk reports, I include every model referenced in the report during model development. Granted this leads to reports that are 1000s of pages long, but every number in the text can be verified by a QC person. So this got me asking myself, what do other companies do? If you don't include the nonmem listing as an appendix and in your text you state that the OFV was X, then how do you reference that number? I'm just curious what others are doing these days? Thanks, pete bonate Peter L. Bonate, PhD Genzyme Corporation Senior Director, Pharmacokinetics 4545 Horizon Hill Blvd San Antonio, TX 78229 USA peter.bonate@genzyme.com phone: 210-949-8662 fax: 210-949-8219

From: "Bill Bachman" bachmanw@comcast.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 10:38:34 -0400 Pete, I havent written a report in a long time (which is a good thing), but, what I had done in the past was to include all SIGNIFICANT models in the report. Of course, SIGNIFICANT, allows a lot of room for interpretation. One would be: base model, full model, and reduced (final) model plus any in which you perhaps tested the influence of some covariate of interest (for effect or lack thereof). Another approach I have taken was to tabulate all models to include structural features, covariates, MVOF to provide a guide to the model building process. All models can be included electronically (archive CD or whatever) but only important models went in the hardcopy. Bill

From: "INTV (Ivan Nestorov)" nestorvi@zgi.com Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 07:44:32 -0700 I do the same as Peter. I include as appendices (hardcopy) the base and final model(s) and any other models that are important, but also provide a CD with all the referenced models in the text as well as other models that have been attempted with the respective data files. I think it would be fair to allow the agency or the other end users to run intermediate runs and to be able to reconstruct anything we refer to while revewing the report. Ivan Nestorov Zymogenetics 1201 Eastlake Avenue East Seattle, Washington 98102 Tel: 206 442 6613 Fax: 206 442 6608 Cell: 425 442 9303 Email: intv@zgi.com

From: "Diane R Mould" drmould@attglobal.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 10:54:50 -0400 Hi All I tend to do the same thing that Bill does I include a model table (actually there are usually several tables for different stages of model development) with as much information as possible in the tables, then control stream and output listing for key runs which usually are base, full and final models. "Lack of effect" is to me a difficult question to address using Nonmem, and therefore if the purpose of the evaluation was to show a lack of effect there would be considerable supportive evaluations to answer that question. If I were only looking at covariates and trying to show that there was no apparent impact of a covariate on the PK, eta plots would be as or more useful than a Nonmem run that shows no improvement in stated criteria Diane

From: "Bill Bachman" bachmanw@comcast.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 11:10:25 -0400 Diane, You are absolutely correct. By "lack of effect", I meant something more like "no apparent impact of a covariate on the PK". Bill

From: David Dai david.dai@bms.com Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 12:09:05 -0400 I practice simlarly to Peter. The main body of my last report is few hundred pages. All important interim model runs are listed in separate tables. The purpose is to show agency the logic line of model building and they can follow it better than only final model is provided. Most models tested will then be listed in the appendix to support main report. The big hassle of writing report like this is the appendix ended up with > 1000 pages. It also increases workload for QC scientist who has to check every number in the main report as well as appendix. What's agency acceptance for a much condensed report? any hint and tips? david -- David Dai, PhD Mail stop: E14-07 Clinical Discovery Bristol-Myers Squibb Company Lawrenceville, NJ 08543 Phone:(609)2526342 Fax:(609)2527821

From: Mark Sale - Next Level Solutions mark@nextlevelsolns.com Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 11:13:19 -0700 Sorry, but I have to weigh in now. Does "the agency" care about the model building logic? Does anyone? or do we only care if the final model passes some set of reasonableness criteria? Wasn't it Thomas Edison who, after several hundred failed attempts at a light bulb said "No, I've found 200 ways not to make a light bulb". IMHO, even when exercised flawlessly, the "logic" that we use to build/select models is inherently flawed, and in my experience, I consistenly find major flaws in the "logic" of my own analyses, let alone someone elses. So, if we are doing this to make (cue scary music) "the agency" happy, should we first ask if they care? (actually I have, but I'll let them speak for themselves). But, as pretty much everyone else I suspect, I make a table of every "relevant" model, with comments, why it was accepted (as current best), or not, and include control files for all "relevant" models in an appendix. Except of course for GA, when I have 6000-10000 models, then I just give a description of the search space and the final model - much shorter report, and can absolutely, alway reproduce the logic - and it is actually logical. So, I'm personally happy to see that the CHMP guidelines don't require the regurgitation of what is probably alway illogical "logic", but focus on the value of the final result. Mark Sale MD Next Level Solutions, LLC www.NextLevelSolns.com

From: "Bill Bachman" bachmanw@comcast.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 14:32:43 -0400 If my memory serves me right, I think the request for logic trail is in the FDA Guidance, for better or worse. Mark, you well know that when the "Agency" says jump, most (but not all) usually say, "How high?" With all due respect to my former colleagues on New Hampshire Avenue! J Bill

From: Mark Sale - Next Level Solutions mark@nextlevelsolns.com Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 12:12:39 -0700 Bill, I'm curious where in the guidance you're refering to. It refers to "significant" step - which I interpret as a fairly strong term - seems likely there might be < 20 "significant" steps in an analaysis. The fact that they suggest a "flow diagram" suggests that no one want to wade through 100's of models. The focus seems to be on discussion of the final model, not how you got there. Here is a section under IX. POPULATION PK STUDY REPORT D. Materials and Methods 3. Data Analysis Methods This section should contain a detailed description of the criteria and procedures for model building and reduction, including exploratory data analysis. The following components of the data analysis method used in the study should be described here: (1) the chosen population analysis method, (2) the assumptions on model components (e.g., parameterization, random effects distributions), (3) the rationale underlying those assumptions, and (4) the chosen model-fitting method. In addition, this section should contain a description of the treatment of outliers and missing data (where applicable), as well as a flow diagram(s) (if possible) of the analysis performed and representative control/command files for each significant model building/reduction step. E. Results The key results of the analysis should be compiled in comprehensible tables and plots. Diagnostic plots used to develop the model and test reliability should be included. To aid interpretation and application, a thorough description of the results should be provided. Complete output of results obtained for the final population model and key intermediate steps should be included. Here is a section describing including the final model in an appendix H. Appendix The appendix should contain a representative portion of the data set used in population analysis. The programming codes along with the printouts of the results of the final model should be included, as well as any additional plots that are deemed important (see section Mark Sale MD Next Level Solutions, LLC www.NextLevelSolns.com

From: "Diane R Mould" drmould@attglobal.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 15:22:03 -0400 Mark I tend to agree that there are many times when "logic" in modeling defies description J That said, I think that trying to explain what led you to a particular model isnt a bad idea. The need to explain this is often dependent on the study / Phase that the data come from. If a model has been developed previously for a NCE, then there should be little need for long descriptions of why models were selected for evaluation. The model should be able to be fairly well specified in the analysis plan. Similarly if an agent is of a particular class of drugs that generally exhibits certain behaviors, then the rationale for model development should be fairly clear, at least in theory Diane

From: "Bill Bachman" bachmanw@comcast.net Subject: RE: [NMusers] Reporting of interim models in Pop PK Reports Date: Tue, 18 Jul 2006 15:47:03 -0400 I think you are arguing circularly, Mark. How do you flow the analysis without discussing how you got there? _______________________________________________________