3Rs: Reproducibility, Replicability and Repeatability

Dear NONMEM Users Does anyone have a clear view on the semantic distinctions between the terms 'reproducibility', 'replicability', and 'repeatability'? There are, surely, different sources of data variation in experiments that might, on the face of it, be identical. However, I have the impression that these terms are being used more often than before, and in a manner that suggests their meanings are transparent and unambiguous. Is there really a consensus out there about what specifically these terms refer to, and is that consensus the same in different scientific disciplines? Gavin __________________________________________________ Dr Gavin E Jarvis MA(Cantab) MA PhD VetMB MRCVS University Lecturer in Veterinary Anatomy Department of Physiology, Development & Neuroscience Physiological Laboratory Downing Street Cambridge CB2 3EG Tel: +44 (0) 1223 333745 Fellow and College Lecturer in Pharmacology Tutor for Graduate Students Selwyn College Cambridge CB3 9DQ Tel: +44 (0) 1223 761303 Email: <mailto:[email protected]> [email protected] Web: http://www.pdn.cam.ac.uk/staff/jarvis www.pdn.cam.ac.uk/staff/jarvis Twitter: @GavinEJarvis

Dear Gavin, My two cents are, in the simplest sense: Reproducibility focuses on the ability to arrive to the same final conclusions, even in the presence of some change, as the 'spirit' of the definition should be about arriving to the same conclusions. (eg use of monolix vs phoenix vs nonmem, or running a second trial) Replicability: ability to recreate the "all" steps and conditions to get to the same results (eg recreate figures/tables from a script, or same in change in objective function value, etc) Repeatability: Whether the same design can be used/achieved again. I think the crux of the issue, is that intra-discipline these can have more distinct definitions, however these are rarely synonymous. One example of these types of semantic clashes is a computer science professor I have spoken with always hated when people used 'real time' when discussing things updating on the web (eg receiving streams of stock market data), because he had worked for years designing nuclear reactors, where real time was measured in fempto seconds, whereas you or I may consider real time within a second or so. I believe the same semantic differences present themselves be said whether discussing assay development, clinical trial design, or modeling & simulation. There could be long debates over small differences in opinion, but I think the most reasonable goal is to land on what the 'spirit' of each focuses on, and choose which term most aligns with your goals. (eg if data qual is the issue, then replicability is important, but if whether a drug works reproducing the outcome should be much more important than simply replicating the same design.) Devin