Usage of parallel nonmem in tendem with SGE

From: Paul Jewell Date: April 28, 2016 technical Source: mail-archive.com
Hello All, I have used nonmem for a while in an environment which uses sun grid engine and mpich2. Currently the two services do not interact. Batch jobs go to the grid engine, and parallel jobs run across all nodes using the mpi daemon. This is usually not an issue, but during times of heavy user activity, the amount of nonmem processes running on each compute node can exceed the total number of cores, causing inefficiency. I am looking for a method of running a parallel job such that it waits for the required number of slots / cores to be available and clear of gridengine jobs before running. (and no new gridengine jobs are submitted after it until it is finished) I have seen that gridengine supports parallel queues but a method of interfacing this with nonmem's parafile specification is not immediately apparent. I wanted to check if there is any possibility of using nonmem/sge in this way before writing a wrapper bash script that does something like this: -submit N number of shell scripts which sleep forever -poll the grid engine until N number of shell scripts are seen running in the queue -begin the parallel run -when done, qdel all of the shell scripts This solution would some problems with evenly using cores, and would require a lot of manual code writing, so I was looking for a better solution first. Please advise if you have heard of any solution. Thank you.
Apr 28, 2016 Paul Jewell Usage of parallel nonmem in tendem with SGE
Apr 28, 2016 Devin Pastoor Re: Usage of parallel nonmem in tendem with SGE
Apr 29, 2016 Ruben Faelens RE: Usage of parallel nonmem in tendem with SGE