Re: CycleCloud BigScience Challenge giving away ~8-hours on 30000 core cluster for research
David,
Some limited benchmarking results are listed at the bottom of this email. It is
also important to remember that there are ways that cloud-computing helps
beyond parallel NONMEM. Cloud-computing allows on demand, user specific,
clusters that can grow and shrink depending on user requirements. This allows
for rapid completion of large bootstrap processes (500 - 1000 jobs) in the time
it takes to run one, non-parallel job. Users can also evaluate model variants
simultaneously with cluster size being increased or decreased as needed. Users
do not have to worry about competing for resources (compute cores) with other
users because cloud-computing clusters are user and/or project specific. The
performance gains are evident in parallel NONMEM jobs and single modeling
projects, but it is important to look beyond the simple benefits of individual
job/project speed-up and more towards the impact of cloud-computing on the
entire portfolio of modeling and simulation projects in a given group or
company.
Bill
Model 1 - ADVAN6, 1000 subjects, dual linear and non-linear elimination
Cores Runtime (hr)
1 80.7
8 16.8
16 8.9
24 6.2
48 3.8
96 2.5
Model 2 - ADVAN6 - 70 subjects, PKPD model
Cores Runtime (hr)
1 4 hr
8 1.1
16 0.66
24 0.4
48 0.31
cores = number of compute cores (value of "NODES=" argument in NONMEM pmn file)
Quoted reply history
On Nov 9, 2011, at 9:28 PM, David Foster wrote:
> I agree with Julia,
>
> Thanks for this Bill, but quantitative benchmarking would be very much
> appreciated.
>
> Regards,
>
> David
>
>
> On 10/11/11 9:55 AM, "Ivashina, Julia" <[email protected]> wrote:
>
> Bill,
>
> This is nice to hear about the speed improvements for NONMEM you received.
>
> Could you please describe gains in performance in a qualitative manner with
> model examples you used.
> I think everyone will benefit from such NONMEM 7.2 benchmarking.
>
> I posted a similar question in March but not many responded.
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On
> Behalf Of Bill Knebel
> Sent: Wednesday, November 09, 2011 5:03 PM
> To: [email protected]
> Cc: Jason Stowe; Nick Holford; William Bachman
> Subject: Re: [NMusers] CycleCloud BigScience Challenge giving away ~8-hours
> on 30000 core cluster for research
>
> Since there seems to be some questions around cloud-based NONMEM, we wanted
> to share our experience of utilizing parallel NONMEM in a cloud-based
> environment. We use the enterprise-licensed version of NONMEM(R) 7.2. to
> perform modeling and simulation, both serial and parallelized, on an AWS
> cloud infrastructure. Briefly, we used a combination of recent MPICH2
> libraries, the Sun Grid Engine integrated ORTE parallel environment, and
> StarCluster to deploy parallel NONMEM(R) on linux AWS EC2 instances of up to
> 100 cores for estimation (which appears to be the current NONMEM(R) 7.2 limit
> for parallel computing), and a virtually unlimited number of cores for
> simulation or use of other open-source modeling tools (R, OpenBUGS). The
> gains in performance are very real, and probably the best we've experienced
> on any multi-user distributed computing infrastructure.
>
> Bill
>
> ~~~~~~~~~~~~~~~~~~~~~~~~
> Bill Knebel, PharmD, PhD
> Principal Scientist II
>
> Metrum Research Group LLC
> 2 Tunxis Road, Suite 112
> Tariffville, CT 06081
>
> O: 860.735.7043
> C: 860.930.1370
> F: 860.760.6014
>
>
> On Nov 4, 2011, at 3:06 PM, Bachman, William wrote:
>
> > Cloud computing on a system like this requires an unlimited enterprise
> > license for NONMEM. Pricing is available from [email protected].
> >
> >
> > William J. Bachman, Ph.D.
> > Director, Pharmacometrics R&D
> > Icon Development Solutions
> > 6031 University Blvd., Suite 300
> > Ellicott City, MD 21043
> > Office 215-616-8699
> > [email protected]
> >
> >
> >
> > From: [email protected] [mailto:[email protected]] On
> > Behalf Of Jason Stowe
> > Sent: Friday, November 04, 2011 2:12 PM
> > To: Nick Holford
> > Cc: [email protected]
> > Subject: Re: [NMusers] CycleCloud BigScience Challenge giving away ~8-hours
> > on 30000 core cluster for research
> >
> > Hi Nick,
> > On Thu, Nov 3, 2011 at 8:59 AM, Nick Holford <[email protected]>
> > wrote:
> > Jason,
> >
> > If I was a winner of your competition how would I go about using NONMEM?
> > How do you manage NONMEM licensing on you cloud cluster?
> > What NONMEM license fee would one have to pay for 30000 cores?
> >
> > Nick
> > Hi Nick,
> > Unfortunately, I can't speak to an individuals user's NONMEM license. We do
> > have commercial clients running NONMEM on CycleCloud clusters at varying
> > scales. A winner with appropriate licensing doesn't need to run 8 hours on
> > 30k cores, you could also run for 24 hours on 10000 cores, etc. If your
> > license relates to compilation vs. execution, or if you have a site-wide
> > license, we can't say, so you'll need to look at your license to deal with
> > that.
> >
> > Perhaps someone from Icon could talk about this kind of non-profit
> > licensing?
> >
> > Thanks,
> > Jason
> >
> >
> >
> > On 2/11/2011 2:19 p.m., Jason Stowe wrote:
> > Hi all,
> >
> > Long time reader, first time poster.
> >
> > Cycle recently announced that we're giving away the equivalent of 8-hrs on
> > a 30000 core CycleCloud HPC cluster for a grand prize winner in the
> > CycleCloud BigScience Challenge. Up to five finalists will win 4 hours on
> > 3000 core clusters as well. The Compute time is to benefit academic and
> > non-profit researchers that are doing science which could benefit humanity.
> >
> > Some current applicants are in life sciences and CycleCloud currently
> > supports reliable/large-scale clusters running NONMEM for Top 5 Pharma, so
> > I wanted to welcome more researchers in the NONMEM community to enter.
> >
> > We describe the challenge and why we're excited about it here:
> > http://bit.ly/BigScience
> >
> > The application process is simply answering 4 questions (takes less than
> > half an hour): State who you are, what is your research, why it is
> > important, and how you currently run computation. The applicatiopn is
> > available here:
> > http://cyclecomputing.com/big-science-challenge/overview
> >
> > So far, response has been great, and Inside HPC covered descriptions of
> > some of the recent applications we've received:
> > http://insidehpc.com/2011/10/27/24209/
> > http://blog.cyclecomputing.com/
> >
> > Submissions are due by November 7th, so submit early and we hope to help
> > some of you get some BigScience done quickly.
> >
> > Best,
> > Jason
> >
> > --
> >
> >
> > ==================================
> > Jason A. Stowe
> > cell: 607.227.9686
> > main: 888.292.5320
> >
> > http://twitter.com/jasonastowe/
> > http://twitter.com/cyclecomputing/
> >
> > Cycle Computing, LLC
> > Leader in Open Compute Solutions for Clouds, Servers, and Desktops
> > Enterprise Condor Support and Management Tools
> >
> > http://www.cyclecomputing.com
> > http://www.cyclecloud.com
> >
> >
> > --
> > Nick Holford, Professor Clinical Pharmacology
> > Dept Pharmacology& Clinical Pharmacology
> > University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
> > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
> > email: [email protected]
> > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
> >
> >
> >
> >
> > --
> >
> >
> > ==================================
> > Jason A. Stowe
> > cell: 607.227.9686
> > main: 888.292.5320
> >
> > http://twitter.com/jasonastowe/
> > http://twitter.com/cyclecomputing/
> >
> > Cycle Computing, LLC
> > Leader in Open Compute Solutions for Clouds, Servers, and Desktops
> > Enterprise Condor Support and Management Tools
> >
> > http://www.cyclecomputing.com
> > http://www.cyclecloud.com
> >
> >