Dear Group,
I would like to get your comments on the following:
Intel fortran on Intel and AMD processors:
I would like to install nonmem compiled using Intel fortran compiler (11.2) in a linux OS with AMD processor. Will the nonmem results of this installation differ from other processors (nonmem compiled using Intel fortran compiler in intel processors).If there would be a difference, what could be the possible reasons for this difference?
OR
Are there any recommendations like optimization flags to get most similar results between these processors?
Thanks for your help
regards,
Martin
Intel vs AMD
Martin Yes, the results can be different. Intel has been accused of "crippling" the executable when the Intel compiler is used on AMD CPUs http://www.agner.org/optimize/blog/read.php?i=49 by turning off all optimization - they actually pretty much admitted this in the lawsuit - but explained that it was for the benefit of the customer - sort of like in the 1980's when Microsoft pretty much disabled WordPerfect with every new OS release. and yes, different optimization setting will give different results, 32 bit will also give different results from 64 bit. Sometimes the phase of the moon, or the users astrological sign makes a different as well ;-) Below is from the settlement: Intel shall not include any Artificial Performance Impairment in any Intel product or require any Third Party to include an Artificial Performance Impairment in the Third Party’s product. As used in this Section 2.3, “ Artificial Performance Impairment ” means an affirmative engineering or design action by Intel (but not a failure to act) that (i) degrades the performance or operation of a Specified AMD product, (ii) is not a consequence of an Intel Product Benefit and (iii) is made intentionally to degrade the performance or operation of a Specified AMD Product. For purposes of this Section 2.3, “ Product Benefit ” shall mean any benefit, advantage, or improvement in terms of performance, operation, price, cost, manufacturability, reliability, compatibility, or ability to operate or enhance the operation of another product. In no circumstances shall this Section 2.3 impose or be construed to impose any obligation on Intel to (i) take any act that would provide a Product Benefit to any AMD or other non-Intel product, either when such AMD or non-Intel product is used alone or in combination with any other product, (ii) optimize any products for Specified AMD Products, or (iii) provide any technical information, documents, or know how to AMD. But, it is generally accepted that Intel continues to impair the optimization on AMD CPU. So, to answer your question, I don't think there is any way to insure consistent results between Intel and AMD CPUs. Mark Mark Sale MD President, Next Level Solutions, LLC www.NextLevelSolns.com 919-846-9185 A carbon-neutral company See our real time solar energy production at: http://enlighten.enphaseenergy.com/public/systems/aSDz2458
Quoted reply history
-------- Original Message --------
Subject: [NMusers] Intel vs AMD
From: Martin Johnson < [email protected] >
Date: Sun, August 12, 2012 3:42 pm
To: [email protected]
Dear Group,
I would like to get your comments on the following:
Intel fortran on Intel and AMD processors:
I would like to install nonmem compiled using Intel fortran compiler
(11.2) in a linux OS with AMD processor. Will the nonmem results of this
installation differ from other processors (nonmem compiled using Intel
fortran compiler in intel processors).If there would be a difference,
what could be the possible reasons for this difference?
OR
Are there any recommendations like optimization flags to get most
similar results between these processors?
Thanks for your help
regards,
Martin
Hi Martin,
At an arbitrary level of precision, there will likely be differences [1].
These differences are unlikely to affect any model results to a notable degree.
When I've done testing previously with different compilers, processors, and
platforms, changing compilers (gcc vs ifortran) made the biggest difference,
and with a stiff problem the differences were <10% of the RSE. You can get
bigger differences than that with the problem I was using just due to initial
theta estimates.
The short answer is that it shouldn't be a problem.
[1] Intel has specifically slowed down their compiler for AMD chips before, but
that is hopefully behind us now.
http://i.downloadsquad.switched.com/2010/01/04/intel-forced-to-provide-a-compiler-that-isnt-crippled-for-amd-processors/
Thanks,
Bill
Quoted reply history
On Aug 12, 2012, at 4:09 PM, "Martin Johnson" <[email protected]> wrote:
> Dear Group,
>
> I would like to get your comments on the following:
>
> Intel fortran on Intel and AMD processors:
>
> I would like to install nonmem compiled using Intel fortran compiler (11.2)
> in a linux OS with AMD processor. Will the nonmem results of this
> installation differ from other processors (nonmem compiled using Intel
> fortran compiler in intel processors).If there would be a difference, what
> could be the possible reasons for this difference?
> OR
> Are there any recommendations like optimization flags to get most similar
> results between these processors?
>
> Thanks for your help
>
> regards,
> Martin
>
Mark,
I think you may not be fully appreciating the terms of the agreement when you say "it is generally accepted that Intel continues to impair the optimization on AMD CPU".Under the terms of the agreement (which you provide below) it is perfectly OK for Intel to optimize their compilers for Intel CPUs without including any optimization for AMD CPUs. Furthermore they are not required to provide any optimization for AMD CPus. Therefore, unless Intel don't know how to optimize compilers you must expect the Intel compiler to perform better on an Intel CPU.
This issue might (or might not) be relevant to Martin's query about 'differences'. He does not specify the kind of difference e.g. faster? more accurate?. It may be possible to choose compiler options that produce identical numerical results on both CPUs but at the price of speed.
Rik Schoemaker suggested using these Intel compiler options /nologo /nbs /w /Gs /fp:strict to obtain consistent numerical results ( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html ) with different NONMEM 7 versions across different operating systems. Perhaps these options would ensure numerical consistency across different CPU types.
Best wishes,
Nick
Quoted reply history
On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
> Martin
>
> Yes, the results can be different. Intel has been accused of "crippling" the executable when the Intel compiler is used on AMD CPUs
>
> http://www.agner.org/optimize/blog/read.php?i=49
>
> by turning off all optimization - they actually pretty much admitted this in the lawsuit - but explained that it was for the benefit of the customer - sort of like in the 1980's when Microsoft pretty much disabled WordPerfect with every new OS release.
>
> and yes, different optimization setting will give different results, 32 bit will also give different results from 64 bit. Sometimes the phase of the moon, or the users astrological sign makes a different as well ;-)
>
> Below is from the settlement:
>
> Intel shall not include any Artificial Performance Impairment in any Intel product or require any Third Party to include an Artificial Performance Impairment in the Third Party’s product. As used in this Section 2.3, “_Artificial Performance Impairment_” means an affirmative engineering or design action by Intel (but not a failure to act) that (i) degrades the performance or operation of a Specified AMD product, (ii) is not a consequence of an Intel Product Benefit and (iii) is made intentionally to degrade the performance or operation of a Specified AMD Product. For purposes of this Section 2.3, “_Product Benefit_” shall mean any benefit, advantage, or improvement in terms of performance, operation, price, cost, manufacturability, reliability, compatibility, or ability to operate or enhance the operation of another product.
>
> In no circumstances shall this Section 2.3 impose or be construed to impose any obligation on Intel to (i) take any act that would provide a Product Benefit to any AMD or other non-Intel product, either when such AMD or non-Intel product is used alone or in combination with any other product, (ii) optimize any products for Specified AMD Products, or (iii) provide any technical information, documents, or know how to AMD.
>
> But, it is generally accepted that Intel continues to impair the optimization on AMD CPU. So, to answer your question, I don't think there is any way to insure consistent results between Intel and AMD CPUs.
>
> Mark
>
> Mark Sale MD
> President, Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
> A carbon-neutral company
> See our real time solar energy production at:
> http://enlighten.enphaseenergy.com/public/systems/aSDz2458
--
Nick Holford, Professor Clinical Pharmacology
First World Conference on Pharmacometrics, 5-7 September 2012
Seoul, Korea http://www.go-wcop.org
Dept Pharmacology & Clinical Pharmacology, Bldg 505 Room 202D
University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
email: [email protected]
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Even when using a non-Intel compiler (e.g. gnu compilers) with exactly the
same compiler options and optimization levels on AMD and Intel that generates
exactly the same
code for both , there can be numerical differences between AMD and Intel
processors.
We did a study [1] on Phoenix NLME results using the gnu g77 compiler with
different Intel processors and Windows operating systems going back to 2002
and up to current generation processors (i3/i5/i7) and Windows 7. We generally
got bit for bit identical results across every system tested when the compiler
settings were the same. But AMD processors gave slightly different results.
I found an AMD technical manual that indicated the probable reason - it advised
that the AMD 80-bit x87 math floating point unit can give slightly different
results than the corresponding Intel FPU.
Indeed, we have found this to be the case in practice.
{1} R. Leary et al, "Exact Reproducibility of Population PK/PD MLME Numerical
Results across Different Computational Environments", PAGE 2011 (abstract 2042].
Quoted reply history
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Nick Holford
Sent: Sunday, August 12, 2012 4:56 PM
To: nmusers
Subject: Re: [NMusers] Intel vs AMD
Mark,
I think you may not be fully appreciating the terms of the agreement when you
say "it is generally accepted that Intel continues to impair the optimization
on AMD CPU".Under the terms of the agreement (which you provide below) it is
perfectly OK for Intel to optimize their compilers for Intel CPUs without
including any optimization for AMD CPUs.
Furthermore they are not required to provide any optimization for AMD CPus.
Therefore, unless Intel don't know how to optimize compilers you must expect
the Intel compiler to perform better on an Intel CPU.
This issue might (or might not) be relevant to Martin's query about
'differences'. He does not specify the kind of difference e.g. faster?
more accurate?. It may be possible to choose compiler options that produce
identical numerical results on both CPUs but at the price of speed.
Rik Schoemaker suggested using these Intel compiler options /nologo /nbs /w /Gs
/fp:strict to obtain consistent numerical results
( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html) with different
NONMEM 7 versions across different operating systems. Perhaps these options
would ensure numerical consistency across different CPU types.
Best wishes,
Nick
On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
> Martin
> Yes, the results can be different. Intel has been accused of
> "crippling" the executable when the Intel compiler is used on AMD CPUs
> http://www.agner.org/optimize/blog/read.php?i=49
>
> by turning off all optimization - they actually pretty much admitted
> this in the lawsuit - but explained that it was for the benefit of the
> customer - sort of like in the 1980's when Microsoft pretty much
> disabled WordPerfect with every new OS release.
>
> and yes, different optimization setting will give different results,
> 32 bit will also give different results from 64 bit. Sometimes the
> phase of the moon, or the users astrological sign makes a different as
> well ;-) Below is from the settlement:
> Intel shall not include any Artificial Performance Impairment in any
> Intel product or require any Third Party to include an Artificial
> Performance Impairment in the Third Party’s product. As used in this
> Section 2.3, “_Artificial Performance Impairment_” means an
> affirmative engineering or design action by Intel (but not a failure
> to act) that (i) degrades the performance or operation of a Specified
> AMD product, (ii) is not a consequence of an Intel Product Benefit and
> (iii) is made intentionally to degrade the performance or operation of
> a Specified AMD Product. For purposes of this Section 2.3, “_Product
> Benefit_” shall mean any benefit, advantage, or improvement in terms
> of performance, operation, price, cost, manufacturability,
> reliability, compatibility, or ability to operate or enhance the
> operation of another product.
>
> In no circumstances shall this Section 2.3 impose or be construed to
> impose any obligation on Intel to (i) take any act that would provide
> a Product Benefit to any AMD or other non-Intel product, either when
> such AMD or non-Intel product is used alone or in combination with any
> other product, (ii) optimize any products for Specified AMD Products,
> or (iii) provide any technical information, documents, or know how to AMD.
>
>
> But, it is generally accepted that Intel continues to impair the
> optimization on AMD CPU.
> So, to answer your question, I don't think there is any way to insure
> consistent results between Intel and AMD CPUs.
>
> Mark
>
>
> Mark Sale MD
> President, Next Level Solutions, LLC
> www.NextLevelSolns.com http://www.NextLevelSolns.com
> 919-846-9185
> A carbon-neutral company
> See our real time solar energy production at:
> http://enlighten.enphaseenergy.com/public/systems/aSDz2458
>
--
Nick Holford, Professor Clinical Pharmacology
First World Conference on Pharmacometrics, 5-7 September 2012 Seoul, Korea
http://www.go-wcop.org
Dept Pharmacology & Clinical Pharmacology, Bldg 505 Room 202D University of
Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
email: [email protected]
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
_________________________________________________________________
Bob, That certainly makes sense, the CPU architechs are constantly tweaking the instruction sets, even for routine things like dividing. So, it is very unlikely that AMD and Intel both came up with exactly the same binary code for even simple math (and binary floating point instructions are always just an approximation to decimal operations). Who can forget the Pentium 5 CPU that had a bug in the instruction set that gave a grossly wrong answer with dividing carefully selection numbers. So, I suspect that CPU type must be added to the things that can cause NONMEM or Phoenix and Monolix to give different results, along with compiler optimization settings 32 vs 64 bit Interestingly, there can (very rarely) even be randomness in the answer. Cosmic rays actually can hit individual bits in memory and flip them. Parity checking/error correcting code (ECC) will detect these single bit changes. The rate is about 10 −10 −10 −17 error/bit·h. Most DRAM in PC is not ECC, most in servers and workstation is ECC. Mark Mark Sale MD President, Next Level Solutions, LLC www.NextLevelSolns.com 919-846-9185 A carbon-neutral company See our real time solar energy production at: http://enlighten.enphaseenergy.com/public/systems/aSDz2458
Quoted reply history
-------- Original Message --------
Subject: RE: [NMusers] Intel vs AMD
From: Bob Leary < [email protected] >
Date: Mon, August 13, 2012 9:13 am
To: Nick Holford < [email protected] >, nmusers
< [email protected] >
Even when using a non-Intel compiler (e.g. gnu compilers) with exactly the same compiler options and optimization levels on AMD and Intel that generates exactly the same
code for both , there can be numerical differences between AMD and Intel processors.
We did a study [1] on Phoenix NLME results using the gnu g77 compiler with different Intel processors and Windows operating systems going back to 2002 and up to current generation processors (i3/i5/i7) and Windows 7. We generally got bit for bit identical results across every system tested when the compiler settings were the same. But AMD processors gave slightly different results. I found an AMD technical manual that indicated the probable reason - it advised that the AMD 80-bit x87 math floating point unit can give slightly different results than the corresponding Intel FPU.
Indeed, we have found this to be the case in practice.
{1} R. Leary et al, "Exact Reproducibility of Population PK/PD MLME Numerical Results across Different Computational Environments", PAGE 2011 (abstract 2042].
-----Original Message-----
From: [email protected] [ mailto: [email protected] ] On Behalf Of Nick Holford
Sent: Sunday, August 12, 2012 4:56 PM
To: nmusers
Subject: Re: [NMusers] Intel vs AMD
Mark,
I think you may not be fully appreciating the terms of the agreement when you say "it is generally accepted that Intel continues to impair the optimization on AMD CPU".Under the terms of the agreement (which you provide below) it is perfectly OK for Intel to optimize their compilers for Intel CPUs without including any optimization for AMD CPUs.
Furthermore they are not required to provide any optimization for AMD CPus. Therefore, unless Intel don't know how to optimize compilers you must expect the Intel compiler to perform better on an Intel CPU.
This issue might (or might not) be relevant to Martin's query about 'differences'. He does not specify the kind of difference e.g. faster?
more accurate?. It may be possible to choose compiler options that produce identical numerical results on both CPUs but at the price of speed.
Rik Schoemaker suggested using these Intel compiler options /nologo /nbs /w /Gs /fp:strict to obtain consistent numerical results
( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html ) with different NONMEM 7 versions across different operating systems. Perhaps these options would ensure numerical consistency across different CPU types.
Best wishes,
Nick
On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
> Martin
> Yes, the results can be different. Intel has been accused of
> "crippling" the executable when the Intel compiler is used on AMD CPUs
> http://www.agner.org/optimize/blog/read.php?i=49
>
> by turning off all optimization - they actually pretty much admitted
> this in the lawsuit - but explained that it was for the benefit of the
> customer - sort of like in the 1980's when Microsoft pretty much
> disabled WordPerfect with every new OS release.
>
> and yes, different optimization setting will give different results,
> 32 bit will also give different results from 64 bit. Sometimes the
> phase of the moon, or the users astrological sign makes a different as
> well ;-) Below is from the settlement:
> Intel shall not include any Artificial Performance Impairment in any
> Intel product or require any Third Party to include an Artificial
> Performance Impairment in the Third Party’s product. As used in this
> Section 2.3, “_Artificial Performance Impairment_” means an
> affirmative engineering or design action by Intel (but not a failure
> to act) that (i) degrades the performance or operation of a Specified
> AMD product, (ii) is not a consequence of an Intel Product Benefit and
> (iii) is made intentionally to degrade the performance or operation of
> a Specified AMD Product. For purposes of this Section 2.3, “_Product
> Benefit_” shall mean any benefit, advantage, or improvement in terms
> of performance, operation, price, cost, manufacturability,
> reliability, compatibility, or ability to operate or enhance the
> operation of another product.
>
> In no circumstances shall this Section 2.3 impose or be construed to
> impose any obligation on Intel to (i) take any act that would provide
> a Product Benefit to any AMD or other non-Intel product, either when
> such AMD or non-Intel product is used alone or in combination with any
> other product, (ii) optimize any products for Specified AMD Products,
> or (iii) provide any technical information, documents, or know how to AMD.
>
>
> But, it is generally accepted that Intel continues to impair the
> optimization on AMD CPU.
> So, to answer your question, I don't think there is any way to insure
> consistent results between Intel and AMD CPUs.
>
> Mark
>
>
> Mark Sale MD
> President, Next Level Solutions, LLC
> www.NextLevelSolns.com < http://www.NextLevelSolns.com >
> 919-846-9185
> A carbon-neutral company
> See our real time solar energy production at:
> http://enlighten.enphaseenergy.com/public/systems/aSDz2458
>
--
Nick Holford, Professor Clinical Pharmacology
First World Conference on Pharmacometrics, 5-7 September 2012 Seoul, Korea http://www.go-wcop.org
Dept Pharmacology & Clinical Pharmacology, Bldg 505 Room 202D University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
email: [email protected]
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
_________________________________________________________________
Thanks for all your comments and now I understand that there could be always slight differences between different CPU/architecture.
However, Would it possible that there could be differences in successful minimizations in nonmem runs. For example, for a same model, in one CPU/architecture the model run is minimized successfully with covariance step and in the other the run is terminated with rounding error.
I would like to say that I have not faced this problem yet. Just a question, to learn from your experience.
regards,
Martin
Quoted reply history
On 08/13/2012 03:13 PM, Bob Leary wrote:
> Even when using a non-Intel compiler (e.g. gnu compilers) with exactly the
> same compiler options and optimization levels on AMD and Intel that generates
> exactly the same
> code for both , there can be numerical differences between AMD and Intel
> processors.
>
> We did a study [1] on Phoenix NLME results using the gnu g77 compiler with
> different Intel processors and Windows operating systems going back to 2002
> and up to current generation processors (i3/i5/i7) and Windows 7. We generally
> got bit for bit identical results across every system tested when the compiler
> settings were the same. But AMD processors gave slightly different results.
> I found an AMD technical manual that indicated the probable reason - it advised
> that the AMD 80-bit x87 math floating point unit can give slightly different
> results than the corresponding Intel FPU.
> Indeed, we have found this to be the case in practice.
>
> {1} R. Leary et al, "Exact Reproducibility of Population PK/PD MLME Numerical
> Results across Different Computational Environments", PAGE 2011 (abstract 2042].
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On
> Behalf Of Nick Holford
> Sent: Sunday, August 12, 2012 4:56 PM
> To: nmusers
> Subject: Re: [NMusers] Intel vs AMD
>
> Mark,
>
> I think you may not be fully appreciating the terms of the agreement when you say
> "it is generally accepted that Intel continues to impair the optimization on AMD
> CPU".Under the terms of the agreement (which you provide below) it is perfectly OK
> for Intel to optimize their compilers for Intel CPUs without including any optimization
> for AMD CPUs.
> Furthermore they are not required to provide any optimization for AMD CPus.
> Therefore, unless Intel don't know how to optimize compilers you must expect
> the Intel compiler to perform better on an Intel CPU.
>
> This issue might (or might not) be relevant to Martin's query about
> 'differences'. He does not specify the kind of difference e.g. faster?
> more accurate?. It may be possible to choose compiler options that produce
> identical numerical results on both CPUs but at the price of speed.
>
> Rik Schoemaker suggested using these Intel compiler options /nologo /nbs /w /Gs
> /fp:strict to obtain consistent numerical results
> ( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html) with different
> NONMEM 7 versions across different operating systems. Perhaps these options
> would ensure numerical consistency across different CPU types.
>
> Best wishes,
>
> Nick
>
> On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
>
> > Martin
> > Yes, the results can be different. Intel has been accused of
> > "crippling" the executable when the Intel compiler is used on AMD CPUs
> > http://www.agner.org/optimize/blog/read.php?i=49
> >
> > by turning off all optimization - they actually pretty much admitted
> > this in the lawsuit - but explained that it was for the benefit of the
> > customer - sort of like in the 1980's when Microsoft pretty much
> > disabled WordPerfect with every new OS release.
> >
> > and yes, different optimization setting will give different results,
> > 32 bit will also give different results from 64 bit. Sometimes the
> > phase of the moon, or the users astrological sign makes a different as
> > well ;-) Below is from the settlement:
> > Intel shall not include any Artificial Performance Impairment in any
> > Intel product or require any Third Party to include an Artificial
> > Performance Impairment in the Third Party’s product. As used in this
> > Section 2.3, “_Artificial Performance Impairment_” means an
> > affirmative engineering or design action by Intel (but not a failure
> > to act) that (i) degrades the performance or operation of a Specified
> > AMD product, (ii) is not a consequence of an Intel Product Benefit and
> > (iii) is made intentionally to degrade the performance or operation of
> > a Specified AMD Product. For purposes of this Section 2.3, “_Product
> > Benefit_” shall mean any benefit, advantage, or improvement in terms
> > of performance, operation, price, cost, manufacturability,
> > reliability, compatibility, or ability to operate or enhance the
> > operation of another product.
> >
> > In no circumstances shall this Section 2.3 impose or be construed to
> > impose any obligation on Intel to (i) take any act that would provide
> > a Product Benefit to any AMD or other non-Intel product, either when
> > such AMD or non-Intel product is used alone or in combination with any
> > other product, (ii) optimize any products for Specified AMD Products,
> > or (iii) provide any technical information, documents, or know how to AMD.
> >
> > But, it is generally accepted that Intel continues to impair the
> > optimization on AMD CPU.
> > So, to answer your question, I don't think there is any way to insure
> > consistent results between Intel and AMD CPUs.
> >
> > Mark
> >
> > Mark Sale MD
> > President, Next Level Solutions, LLC
> > http://www.NextLevelSolns.com
> > 919-846-9185
> > A carbon-neutral company
> > See our real time solar energy production at:
> > http://enlighten.enphaseenergy.com/public/systems/aSDz2458
>
> --
> Nick Holford, Professor Clinical Pharmacology
>
> First World Conference on Pharmacometrics, 5-7 September 2012 Seoul, Korea
> http://www.go-wcop.org
>
> Dept Pharmacology& Clinical Pharmacology, Bldg 505 Room 202D University of
> Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
> email: [email protected]
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
>
> _________________________________________________________________
>
Martin,
In my experience NONMEM internally uses a pseudo-random process to decide if convergence has been achieved even when the objective function and parameter estimates are essentially the same. As far as I know this has nothing to do with the compiler or CPU. So I do not pay much attention to NONMEM's declaration of success.
Nick
Quoted reply history
On 14/08/2012 9:00 a.m., Martin Johnson wrote:
> Thanks for all your comments and now I understand that there could be always slight differences between different CPU/architecture.
>
> However, Would it possible that there could be differences in successful minimizations in nonmem runs. For example, for a same model, in one CPU/architecture the model run is minimized successfully with covariance step and in the other the run is terminated with rounding error.
>
> I would like to say that I have not faced this problem yet. Just a question, to learn from your experience.
>
> regards,
> Martin
>
> On 08/13/2012 03:13 PM, Bob Leary wrote:
>
> > Even when using a non-Intel compiler (e.g. gnu compilers) with exactly the same compiler options and optimization levels on AMD and Intel that generates exactly the same code for both , there can be numerical differences between AMD and Intel processors.
> >
> > We did a study [1] on Phoenix NLME results using the gnu g77 compiler with different Intel processors and Windows operating systems going back to 2002 and up to current generation processors (i3/i5/i7) and Windows 7. We generally got bit for bit identical results across every system tested when the compiler settings were the same. But AMD processors gave slightly different results. I found an AMD technical manual that indicated the probable reason - it advised that the AMD 80-bit x87 math floating point unit can give slightly different results than the corresponding Intel FPU.
> >
> > Indeed, we have found this to be the case in practice.
> >
> > {1} R. Leary et al, "Exact Reproducibility of Population PK/PD MLME Numerical Results across Different Computational Environments", PAGE 2011 (abstract 2042].
> >
> > -----Original Message-----
> >
> > From: [email protected] [ mailto: [email protected] ] On Behalf Of Nick Holford
> >
> > Sent: Sunday, August 12, 2012 4:56 PM
> > To: nmusers
> > Subject: Re: [NMusers] Intel vs AMD
> >
> > Mark,
> >
> > I think you may not be fully appreciating the terms of the agreement when you say "it is generally accepted that Intel continues to impair the optimization on AMD CPU".Under the terms of the agreement (which you provide below) it is perfectly OK for Intel to optimize their compilers for Intel CPUs without including any optimization for AMD CPUs. Furthermore they are not required to provide any optimization for AMD CPus. Therefore, unless Intel don't know how to optimize compilers you must expect the Intel compiler to perform better on an Intel CPU.
> >
> > This issue might (or might not) be relevant to Martin's query about 'differences'. He does not specify the kind of difference e.g. faster? more accurate?. It may be possible to choose compiler options that produce identical numerical results on both CPUs but at the price of speed.
> >
> > Rik Schoemaker suggested using these Intel compiler options /nologo /nbs /w /Gs /fp:strict to obtain consistent numerical results ( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html ) with different NONMEM 7 versions across different operating systems. Perhaps these options would ensure numerical consistency across different CPU types.
> >
> > Best wishes,
> >
> > Nick
> >
> > On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
> >
> > > Martin
> > > Yes, the results can be different. Intel has been accused of
> > > "crippling" the executable when the Intel compiler is used on AMD CPUs
> > > http://www.agner.org/optimize/blog/read.php?i=49
> > >
> > > by turning off all optimization - they actually pretty much admitted
> > > this in the lawsuit - but explained that it was for the benefit of the
> > > customer - sort of like in the 1980's when Microsoft pretty much
> > > disabled WordPerfect with every new OS release.
> > >
> > > and yes, different optimization setting will give different results,
> > > 32 bit will also give different results from 64 bit. Sometimes the
> > > phase of the moon, or the users astrological sign makes a different as
> > > well ;-) Below is from the settlement:
> > > Intel shall not include any Artificial Performance Impairment in any
> > > Intel product or require any Third Party to include an Artificial
> > > Performance Impairment in the Third Party’s product. As used in this
> > > Section 2.3, “_Artificial Performance Impairment_” means an
> > > affirmative engineering or design action by Intel (but not a failure
> > > to act) that (i) degrades the performance or operation of a Specified
> > > AMD product, (ii) is not a consequence of an Intel Product Benefit and
> > > (iii) is made intentionally to degrade the performance or operation of
> > > a Specified AMD Product. For purposes of this Section 2.3, “_Product
> > > Benefit_” shall mean any benefit, advantage, or improvement in terms
> > > of performance, operation, price, cost, manufacturability,
> > > reliability, compatibility, or ability to operate or enhance the
> > > operation of another product.
> > >
> > > In no circumstances shall this Section 2.3 impose or be construed to
> > > impose any obligation on Intel to (i) take any act that would provide
> > > a Product Benefit to any AMD or other non-Intel product, either when
> > > such AMD or non-Intel product is used alone or in combination with any
> > > other product, (ii) optimize any products for Specified AMD Products,
> > >
> > > or (iii) provide any technical information, documents, or know how to AMD.
> > >
> > > But, it is generally accepted that Intel continues to impair the
> > > optimization on AMD CPU.
> > > So, to answer your question, I don't think there is any way to insure
> > > consistent results between Intel and AMD CPUs.
> > >
> > > Mark
> > >
> > > Mark Sale MD
> > > President, Next Level Solutions, LLC
> > > http://www.NextLevelSolns.com
> > > 919-846-9185
> > > A carbon-neutral company
> > > See our real time solar energy production at:
> > > http://enlighten.enphaseenergy.com/public/systems/aSDz2458
> >
> > --
> > Nick Holford, Professor Clinical Pharmacology
> >
> > First World Conference on Pharmacometrics, 5-7 September 2012 Seoul, Korea http://www.go-wcop.org
> >
> > Dept Pharmacology& Clinical Pharmacology, Bldg 505 Room 202D University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
> >
> > tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
> > email: [email protected]
> > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
> >
> > _________________________________________________________________
> >
> >
Dear Martin,
Yes, you can certainly get a succesful minimisation on one platform and
rounding errors on the other; not consistently fortunately or unfortunately
though :-).
The settings that Nick quoted ("Rik Schoemaker suggested using these Intel
compiler options /nologo /nbs /w /Gs /fp:strict to obtain consistent numerical
results ( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html) with
different NONMEM 7 versions across different operating systems. Perhaps these
options would ensure numerical consistency across different CPU types."), do
ensure exactly identical results on 64 bit OS platforms (Linux, Windows, OSX)
with FO and FOCE methods for Intel Visual Fortran 11.1 when using Intel chips
but results will not be exactly the same when using AMD chips.
In terms of science, differences are not an issue (and down to the Nick's
random number properties), but they are of course a total nuisance when you
want reproducibility.
Cheers,
Rik Schoemaker, PhD
Exprimo NV
Tel: +31 (0)20 4416410
E-mail: [email protected]
Web: www.exprimo.com
This e-mail is confidential. It is also privileged or otherwise protected by
work product immunity or other legal rules. The information is intended to be
for use of the individual or entity named above. If you are not the intended
recipient, please be aware that any disclosure, copying, distribution or use of
the contents of this information is prohibited. You should therefore delete
this message from your computer system. If you have received the message in
error, please notify us by reply e-mail. The integrity and security of this
message cannot be guaranteed on the Internet.
Thank you for your co-operation.
Quoted reply history
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Martin Johnson
Sent: 13 August 2012 11:01 PM
To: Bob Leary
Cc: Nick Holford; nmusers
Subject: Re: [NMusers] Intel vs AMD
Thanks for all your comments and now I understand that there could be always
slight differences between different CPU/architecture.
However, Would it possible that there could be differences in successful
minimizations in nonmem runs. For example, for a same model, in one
CPU/architecture the model run is minimized successfully with covariance step
and in the other the run is terminated with rounding error.
I would like to say that I have not faced this problem yet. Just a question, to
learn from your experience.
regards,
Martin
On 08/13/2012 03:13 PM, Bob Leary wrote:
> Even when using a non-Intel compiler (e.g. gnu compilers) with
> exactly the same compiler options and optimization levels on AMD and Intel
> that generates exactly the same code for both , there can be numerical
> differences between AMD and Intel processors.
>
> We did a study [1] on Phoenix NLME results using the gnu g77 compiler with
> different Intel processors and Windows operating systems going back to 2002
> and up to current generation processors (i3/i5/i7) and Windows 7. We
> generally got bit for bit identical results across every system tested when
> the compiler settings were the same. But AMD processors gave slightly
> different results. I found an AMD technical manual that indicated the
> probable reason - it advised that the AMD 80-bit x87 math floating point
> unit can give slightly different results than the corresponding Intel FPU.
> Indeed, we have found this to be the case in practice.
>
>
> {1} R. Leary et al, "Exact Reproducibility of Population PK/PD MLME
> Numerical Results across Different Computational Environments", PAGE 2011
> (abstract 2042].
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Nick Holford
> Sent: Sunday, August 12, 2012 4:56 PM
> To: nmusers
> Subject: Re: [NMusers] Intel vs AMD
>
> Mark,
>
> I think you may not be fully appreciating the terms of the agreement when you
> say "it is generally accepted that Intel continues to impair the optimization
> on AMD CPU".Under the terms of the agreement (which you provide below) it is
> perfectly OK for Intel to optimize their compilers for Intel CPUs without
> including any optimization for AMD CPUs.
> Furthermore they are not required to provide any optimization for AMD CPus.
> Therefore, unless Intel don't know how to optimize compilers you must expect
> the Intel compiler to perform better on an Intel CPU.
>
> This issue might (or might not) be relevant to Martin's query about
> 'differences'. He does not specify the kind of difference e.g. faster?
> more accurate?. It may be possible to choose compiler options that produce
> identical numerical results on both CPUs but at the price of speed.
>
> Rik Schoemaker suggested using these Intel compiler options /nologo
> /nbs /w /Gs /fp:strict to obtain consistent numerical results
> ( http://www.cognigencorp.com/nonmem/current/2011-May/3266.html) with
> different NONMEM 7 versions across different operating systems. Perhaps these
> options would ensure numerical consistency across different CPU types.
>
> Best wishes,
>
> Nick
>
> On 13/08/2012 8:27 a.m., Mark Sale - Next Level Solutions wrote:
>> Martin
>> Yes, the results can be different. Intel has been accused of
>> "crippling" the executable when the Intel compiler is used on AMD
>> CPUs
>> http://www.agner.org/optimize/blog/read.php?i=49
>>
>> by turning off all optimization - they actually pretty much admitted
>> this in the lawsuit - but explained that it was for the benefit of
>> the customer - sort of like in the 1980's when Microsoft pretty much
>> disabled WordPerfect with every new OS release.
>>
>> and yes, different optimization setting will give different results,
>> 32 bit will also give different results from 64 bit. Sometimes the
>> phase of the moon, or the users astrological sign makes a different
>> as well ;-) Below is from the settlement:
>> Intel shall not include any Artificial Performance Impairment in any
>> Intel product or require any Third Party to include an Artificial
>> Performance Impairment in the Third Party’s product. As used in this
>> Section 2.3, “_Artificial Performance Impairment_” means an
>> affirmative engineering or design action by Intel (but not a failure
>> to act) that (i) degrades the performance or operation of a Specified
>> AMD product, (ii) is not a consequence of an Intel Product Benefit
>> and
>> (iii) is made intentionally to degrade the performance or operation
>> of a Specified AMD Product. For purposes of this Section 2.3,
>> “_Product Benefit_” shall mean any benefit, advantage, or improvement
>> in terms of performance, operation, price, cost, manufacturability,
>> reliability, compatibility, or ability to operate or enhance the
>> operation of another product.
>>
>> In no circumstances shall this Section 2.3 impose or be construed to
>> impose any obligation on Intel to (i) take any act that would provide
>> a Product Benefit to any AMD or other non-Intel product, either when
>> such AMD or non-Intel product is used alone or in combination with
>> any other product, (ii) optimize any products for Specified AMD
>> Products, or (iii) provide any technical information, documents, or know how
>> to AMD.
>>
>>
>> But, it is generally accepted that Intel continues to impair the
>> optimization on AMD CPU.
>> So, to answer your question, I don't think there is any way to insure
>> consistent results between Intel and AMD CPUs.
>>
>> Mark
>>
>>
>> Mark Sale MD
>> President, Next Level Solutions, LLC
>> http://www.NextLevelSolns.com
>> 919-846-9185
>> A carbon-neutral company
>> See our real time solar energy production at:
>> http://enlighten.enphaseenergy.com/public/systems/aSDz2458
>>
>
> --
> Nick Holford, Professor Clinical Pharmacology
>
> First World Conference on Pharmacometrics, 5-7 September 2012 Seoul,
> Korea http://www.go-wcop.org
>
> Dept Pharmacology& Clinical Pharmacology, Bldg 505 Room 202D
> University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New
> Zealand
> tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
> email: [email protected]
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
>
>
>
> _________________________________________________________________
>