freezes during FOCE+I search with nm743 on Linux

From: Ka Ho Hui Date: December 28, 2018 technical Source: mail-archive.com
Dear NMusers, Recently we are encountering freezes during FOCE+I searches on our Linux system. The version of NONMEM used is 7.4.3. We set $EST PRINT=1 and observed that for some datasets (we are working on a large number of simulated datasets) the search would halt indefinitely at certain iteration no. without any further progress (so, never terminated). The only file being constantly updated was the .cpu file. We resorted to strace and observed that at such iteration no., the following block popped up repeatedly: wait4(5956, 0x7ffc2e280874, WNOHANG, NULL) = 0 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 nanosleep({1, 0}, {1, 0}) = 0 But we are not sure what to do with it. Other observations: * The exact same model file and datafile terminated successfully with nm743 on a Windows 10 system. * Whenever this problem occured there seemed to be zero gradients going on. * We suspected handling of arithmetic errors such as div/0 error and implemented something like this under $ERROR, but it did not resolve the halt: FLAG=0 IF (F.EQ.0) FLAG=1 Y=((1-FLAG)*F+FLAG*0.001)*(1+EPS(1)) * Supplying lower and upper bounds on relevant theta('s) seems to remove the halt. We are wondering if this is a bug in the Linux version. Even if convergence cannot be achieved, we think that the search process needs to timeout itself. Any suggestion? P.S. Please let me know if an example set of control stream and datafile is desired. Thanks and regards, Matthew