Re: unscientific poll

Thanks Leonid for this service to our community! I am not surprised by the results but rather by the number of replies. There are supposedly 1000 NONMEM users on this beautiful planet, and 35 participated in the survey. My guess is that these are about 50% of the active participants of the usernet. Therefore we may have, let's say, 100 regular readers of the usernet. That is 10% of the users. That number troubles me as it tells me that 90% of the NONMEM users have established their routine and see little need (have little time) to read the postings of the usernet. Yet, the discussions on our usernet are extremely relevant to our daily modeling work. Cheers to our usernet! Joachim __________________________________________ Joachim GREVEL, Ph.D. MERCK SERONO International S.A. Exploratory Medicine 1202 Geneva Tel: +41.22.414.4751 Fax: +41.22.414.3059 Email: joachim.grevel erono.net Leonid Gibiansky <LGibiansky /2008 07:00 PM To nmusers <nmusers c Subject [NMusers] unscientific poll for the questions that I sent out recently. I received a total of 35 replies. This e-mail consists of 4 parts, as follows: Part 1: Each original questions is followed by the summary of replies Part 2: All comments that I received are copy-pasted after the summary of replies. Part 3: CSV file with the original data is copy-pasted after the comments Part 4: R code that I used to summarize the results is provided Thanks to all who participated. Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 ############ RESULTS ########################### syntax) > error/warning messages (134, 137, number of significant digits, etc.) > and "MINIMIZATION SUCCESSFUL" messages (YES/NO): question YES No Missing Q1 2 (5.7%) 33 (94.3%) 0 (0%) > 2. Do you remember at least one example when the run-time error message helped you to find an error in your code (YES/NO): question YES No MissingQ2 31 (88.6%) 3 (8.6%) 1 (2.9%) > 3. In your experience, run-time error messages allow you to detect model errors or problems quicker than it would be done without error messages: (agree/disagree) question AGREE No Missing Q3 27 (77.1%) 3 (8.6%) 5 (14.3%) > 4. Have you ever used in your report/publication ANY model that did not have $COV step completed (YES/NO): question YES No Missing Q4 25 (71.4%) 9 (25.7%) 1 (2.9%)> 5. Have you ever used in your report/publication ANY model that did not converge (YES/NO): question YES No Missing Q5 13 (37.1%) 21 (60%) 1 (2.9%) that did > not have $COV step completed (YES/NO): question YES No Missing Q6 16 (45.7%) 18 (51.4%) 1 (2.9%) > 7. Have you ever used in your report/publication FINAL model that did > not converge (YES/NO): question YES No Missing Q7 3 (8.6%) 31 (88.6%) 1 (2.9%) > 8. Define yourself as novice/intermediate/experienced Nonmem user: Missing Novice Intermediate Experienced 1 3 15 16 ############ Comments ########################### Honestly, nobody is proposing to remove the minimization successful statement or any non syntax error message. I agree with the comments that stress it's important to take them with a pinch of salt as these "errors" not always point you in the right directions, and not always being picky about $COV step or number of significant digits help in selecting the best model------------------------- Question 1 has at least 3 parts and cannot be answered YES or NO in any meaningful way. Please note the messages we have discussed are not ERROR or WARNING messages. They are a message about the minimization status. I would be happy if NONMEM stuck to the facts. It can tell me if it achieved the requested sigdigs (CONVERGED) or ran out of function evals (PREMATURE TERMINATION). But it should keep its subjective judgements to itself. If you made a list of run-time error messages and another of run-time warning messages then perhaps your survey could be more helpful in deciding which are meaningful? ---------------------- My 2 cents on the discussion, for what it?s worth? The error message is usually related to an error in the dataset / dataset programming issue / initial estimates / model parameterization. Generally, all need to be addressed. Just as important, I wouldn?t trust a MINIMIZATION SUCCESSFUL message either as you will often see flip-flop or 3-CMT identifiability issues that NONMEM doesn?t flag. NONMEM has its flaws, but I?ve seen many of NONMEM?s error messages disappear after correcting a programming error in the dataset, refining the initial estimates, or re-parameterizing the model. I think the practical pharmacometrician isn?t going to accept NONMEM outputs strictly at face value, but would challenge the findings to confirm the results. That?s just good science. ---------------------- nverge or failed the COV step. My guess is that individuals who frequently do probably tend to be more mechanistic in their model building than I am and often push the complexity of their models beyond what can be supported by the data in hand. For those that do report out models that don't converge, I wonder if they have tried re-running their models with different starting values (15-20% different) and see if NONMEM fails to converge at the same set of parameter estimates. My guess is in many cases it won't although both sets of estimates may appear "reasonable" and give similar fits and VPC. models, my thinking is that rather than using approximate maximum model is unstable or not fully supported by the data, I think they would be better served by using a Bayesian approach. That way they can be explicit about the strength of their priors and they don't have to worry about convergence and COV step failures. JMHO. ------------------ ############## Original Data ################## ID,Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8 1,0,0,-1,1,1,1,0,1 ,0,0,-1,1,0,1,0,2 6,0,1,-1,1,0,1,0,3 7,0,1,1,0,0,0,0,3 8,1,1,0,0,0,0,0,3 9,0,1,1,0,0,0,0,2 10,0,1,1,1,0,0,0,-1 11,0,1,1,1,0,1,0,3 12,0,0,1,0,0,0,0,2 13,0,1,1,1,1,1,0,2 14,0,1,1,0,0,0,0,3 15,0,1,1,1,1,1,0,3 16,1,-1,0,1,1,1,0,2 17,0,1,1,-1,-1,-1,-1,1 18,0,1,1,0,0,0,0,3 19,0,1,1,1,0,0,0,3 20,0,1,1,1,0,0,0,3 21,0,1,1,1,1,0,0,1 22,0,1,1,0,0,0,0,2 23,0,1,0,1,1,1,0,3 24,0,1,1,1,1,0,0,2 25,0,1,1,1,1,0,0,3 26,0,1,1,0,0,0,0,227,0,1,1,1,0,0,0,2 28,0,1,1,1,1,1,0,2 29,0,1,1,1,0,1,0,3 33,0,1,1,1,1,1,0,2 34,0,1,1,1,0,0,0,2 35,0,1,1,0,0,0,0,2 .data <- read.table("C:/poll.csv",sep=",",header=T) res <- NULL for(Qname in paste("Q",1:7,sep="") ){ x <- raw.data[,Qname] n.yes <- sum(x = 1) n.no <- sum(x = = 0) n.na <- sum(x = -1) n <- length(x) temp <- data.frame(n=n,question=Qname, YES=paste(n.yes," (",round(100*n.yes/n,1),"%)",sep=""), No=paste(n.no," (",round(100*n.no/n,1),"%)",sep=""), 1),"%)",sep="")) res <- rbind(res,temp) } res table(raw.data$Q8) ########################################################## ----------------------------------------- This message and any attachment are confidential, may be privileged or otherwise protected from disclosure and are intended only for use by the addressee(s) named herein. If you are not the intended recipient, you must not copy this message or attachment or disclose sion in error, please notify the sender immediately and delete the message and any attachment from your system. Merck Serono does not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. If verification is required, please request a hard-copy version. Merck Serono does not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith. --=_alternative 00273275C1257515_=--