Re: Validation Strategy for NONMEM

On Tue, Oct 21, 2008 at 4:11 PM, Nick Holford <[EMAIL PROTECTED]>wrote: > Tony, > > To expand a bit on your useful validation definition. > > Making sure computational results are reproducible does not mean they are > correct. I believe that most NONMEM analyses cannot be proven to be correct > therefore the validation efforts only assure reproducibility. Whether this > makes the validation useful or useless is debatable. > > Nick > > > > > A.J. Rossini wrote: > >> There is useless validation, and then there is useful validation. The >> latter is about making sure your computational results are >> reproducible, the former is about making sure that your documentation >> can be photocopied. It's sort of the same thing, if you aren't a >> modeler. >> >> Unfortunately, most on this list tend to be modelers. >> >> On Tue, Oct 21, 2008 at 7:37 PM, <[EMAIL PROTECTED]> wrote: >> >> >>> After reading this, it is no wonder scientific productivity is at an >>> all-time low. Imagine if Marie Curie had to qualify her radium-purifying >>> equipment, or if Alexander Fleming had to validate his petri dishes >>> before >>> culturing Penecillium. One day scientists are going to push back against >>> these IT people, who just make busy work for everyone. >>> >>> >>> >>> [EMAIL PROTECTED] >>> Sent by: [EMAIL PROTECTED] >>> >>> 21-Oct-2008 03:18 >>> >>> >>> To >>> [email protected] >>> cc >>> Subject >>> Re: [NMusers] Validation Strategy for NONMEM >>> >>> >>> >>> >>> >>> Dear Mark, >>> >>> I am engaged in this question since the beginning of this year (not >>> finished >>> yet), and I am happy to share some basics of my experiences: >>> >>> 1. It is important to specify where the data for NONMEM analysis >>> come >>> from. If they come from a GCP source and are already QA-ed when they >>> arrive >>> at the doorstep of NONMEM then your system will only subject to GCP >>> regulation. Otherwise, you also have to comply with GLP. >>> >>> 2. There is no way around what is called here a 'Validation Plan' >>> and >>> a 'Risk Analysis'. These documents will trigger a slate of other >>> documents >>> (in our case here about 15) which describe Installation, Installation >>> Validation, Qualification of Users, Modeling Strategy, Review Processes, >>> System Life Cycle Management etc. >>> >>> 3. We found it useful to differentiate between 'Exploratory Work' >>> and >>> 'Submission Work'. >>> >>> 4. Before you worry about passing inspection by the FDA, you need >>> to >>> worry about passing inspection of your own company QA officers. >>> >>> 5. Just installing NONMEM with NMQual does not render you new >>> system >>> 'validated' or 'qualified'. Here my apologies to the excellent folks at >>> Metrum, but for various reasons, we ended up not using NMQual. >>> >>> 6. You have to know what you are trying to build before you >>> concern >>> yourself about QA processes. A number of separate installations on PCs >>> linked to a file server is a different animal from a server-based >>> installation with a grid engine. >>> >>> 7. It all takes more time than you think: make generous budgets >>> and >>> time lines. >>> >>> I hope I helped more than I confused, >>> >>> Joachim >>> >>> __________________________________________ >>> Joachim GREVEL, Ph.D. >>> MERCK SERONO International S.A. >>> Exploratory Medicine >>> 1202 Geneva >>> Tel: +41.22.414.4751 >>> Fax: +41.22.414.3059 >>> Email: [EMAIL PROTECTED] >>> >>> >>> >>> "Vilicich, Mark" <[EMAIL PROTECTED]> >>> Sent by: [EMAIL PROTECTED] >>> >>> 10/17/2008 10:08 PM >>> >>> To >>> <[email protected]> >>> cc >>> Subject >>> [NMusers] Validation Strategy for NONMEM >>> >>> >>> >>> >>> >>> >>> Dear All, >>> >>> I am interested in perspectives on strategies for "validating" NONMEM. >>> Also, >>> experiences from or with the FDA since the FDA is: a key user, customer >>> of >>> analysis and auditor of NONMEM use in the industry. Without a large >>> nonmem >>> staff here, the challenge I see is in scaling the validation strategy to >>> provide the most efficient environment for doing analysis that is >>> defensible >>> to both internal and external audits based on the associated GxP risk >>> level. >>> >>> Below are the concepts I've cobbled together, though instead of my >>> reinventing the wheel I appreciate anything you could share. Any and all >>> gems of insight you can share whether it regard the big picture or some >>> detailed specifics, IT centric or business process related. You may send >>> them back to the listserver or me directly as you feel appropriate. >>> >>> Details: >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> From searching the archives and other random bits of knowledge on NONMEM, >>> part of the validation strategy is to recognize that NONMEM is not to be >>> literally validated. NONMEM may be considered more of a development >>> environment, optimized for developing specialized forms of complex >>> analysis >>> and modeling. As a development platform, an approach could be that NONMEM >>> itself is qualified and each specific analysis is validated individually. >>> >>> To support establishing a defensible NONMEM environment, I've also read >>> discussions on integrating common software development best practices >>> such >>> as version control of the "programming" of nonmem, NMQual and other >>> commercial and custom tools for capturing all the metadata related to >>> running a specific NONMEM job. These themes support defining the state of >>> the NONMEM environment and ability to reproduce the outcomes. >>> >>> Also, reading in the archives about the differences in the numeric >>> outcomes >>> of NONMEM analyses based on the hardware platform, etc. are helpful to >>> know >>> up front and to consider in the validation strategy so it is not destined >>> to >>> failure if the target environment is multiplatform or otherwise complex. >>> >>> Gaps noticed/topics not discussed: >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> Is there opportunity in looking at the risk based approached sanctioned >>> by >>> the FDA a few years ago that would make the total validation deliverable, >>> including both the application and the model development process, more >>> lean >>> and targeted at the primary risk targets? >>> >>> Does this scientific software environment lend itself to use of modern >>> agile >>> software development methodologies that go far beyond basic iterative >>> approaches. These methodologies are being used in software development >>> for >>> the regulated/GxP industry. >>> >>> I've seen the excellent presentation from 2004 that Joga Gobburu from the >>> FDA gave, seems like there has been some progression of thought or >>> actions >>> on the proposals included there. Any references to follow-up information >>> on >>> it would be helpful? >>> >>> Regards , >>> >>> Mark Vilicich >>> Early Development >>> [EMAIL PROTECTED] >>> >>> ________________________________ >>> >>> This message and any attachment are confidential, may be privileged or >>> otherwise protected from disclosure and are intended only for use by the >>> addressee(s) named herein. If you are not the intended recipient, you >>> must >>> not copy this message or attachment or disclose the contents to any other >>> person. If you have received this transmission in error, please notify >>> the >>> sender immediately and delete the message and any attachment from your >>> system.