Dear NONMEM users,
Currently I am working on the simulation of a bio-equavalence trial. For the reference compound a population PK model has been derived on basis of data from 100 patients. Values for between-and within-patient variability are available for all PK parameters. The simulation comprises a randomized cross-over study with 12 patients taking the test and reference compound. Two-hunderd trials are simulated and summarized. During the simulations I noticed that truncation of the simulated of PK parameters significantly influences the power of the study to confirm bio-equivalence. For instance truncation of simulated oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to decreased within- patient variability). Post-hoc estimates form the popPK study indicated that clearance values of the reference compound are all within the latter range of 5 to 30 L/hr. I expect that oral clearance of the test compound will not differ more than 5% from the reference compound. In my opinion simulation of trials with the smallest range will produce more reliable estimates of the power to detect bio-equivalence.
I would greatly appreciate your comments on this subject.
Best regards,
Ron Mathôt
Department of Hospital Pharmacy and Clincal Pharmacology
Erasmus University Medical Center
Rotterdam
The Netherlands
truncation & simulation
Ron,
When you truncate the simulated parameter distribution it can lead to a major violation of the assumptions of maximum likelihood i.e. that all random effects are normally distributed. This means that the likelihood ratio test will have a larger Type 1 error than expected from using the chi-2 distribution assumption. You should use a randomization test in order to determine what change in OFV is needed in order to reject the null under your desired hypothesis.
Nick
Ron Mathôt wrote:
> Dear NONMEM users,
>
> Currently I am working on the simulation of a bio-equavalence trial. For the reference compound a population PK model has been derived on basis of data from 100 patients. Values for between-and within-patient variability are available for all PK parameters. The simulation comprises a randomized cross-over study with 12 patients taking the test and reference compound. Two-hunderd trials are simulated and summarized. During the simulations I noticed that truncation of the simulated of PK parameters significantly influences the power of the study to confirm bio-equivalence. For instance truncation of simulated oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to decreased within- patient variability). Post-hoc estimates form the popPK study indicated that clearance values of the reference compound are all within the latter range of 5 to 30 L/hr. I expect that oral clearance of the test compound will not differ more than 5% from the reference compound. In my opinion simulation of trials with the smallest range will produce more reliable estimates of the power to detect bio-equivalence.
>
> I would greatly appreciate your comments on this subject.
> Best regards,
>
> Ron Mathôt
>
> Department of Hospital Pharmacy and Clincal Pharmacology
> Erasmus University Medical Center
> Rotterdam
> The Netherlands
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)373-7599x86730 fax:+64(9)373-7090
www.health.auckland.ac.nz/pharmacology/staff/nholford
Ron
I guess, the model over-estimates variances of the random effects (could be due to non-normality of the distributions or some outliers in the data).
Instead of truncation, I would suggest to sample from the 100 subjects that you have in your dataset:
For each study, randomly select 12 subjects (I would do it without replacement) out of 100 that you have in your dataset. Use the PK parameters of those subjects for simulations. If you expect population of the BE trial to differ from the population of the trial that you had in your model, use ETAs of those 100 patients (correlated, as a vector) instead of PK parameters, and compute PK parameters based on the expected patient covariates.
Leonid
--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566
Ron Mathôt wrote:
> Dear NONMEM users,
>
> Currently I am working on the simulation of a bio-equavalence trial. For the reference compound a population PK model has been derived on basis of data from 100 patients. Values for between-and within-patient variability are available for all PK parameters. The simulation comprises a randomized cross-over study with 12 patients taking the test and reference compound. Two-hunderd trials are simulated and summarized. During the simulations I noticed that truncation of the simulated of PK parameters significantly influences the power of the study to confirm bio-equivalence. For instance truncation of simulated oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to decreased within- patient variability). Post-hoc estimates form the popPK study indicated that clearance values of the reference compound are all within the latter range of 5 to 30 L/hr. I expect that oral clearance of the test compound will not differ more than 5% from the reference compound. In my opinion simulation of trials with the smallest range will produce more reliable estimates of the power to detect bio-equivalence.
>
> I would greatly appreciate your comments on this subject.
> Best regards,
>
> Ron Mathôt
>
> Department of Hospital Pharmacy and Clincal Pharmacology
> Erasmus University Medical Center
> Rotterdam
> The Netherlands
Ron,
Truncation can also introduce bias so you should check for this in your
simulations as well. Also, if your model based on 100 patients results in
simulations of CL/F ranging from 1 300 L/hr but the post hoc estimates of
CL/F range from 5 30 L/hr then you may want to further assess the
appropriateness of your model before using it to conduct clinical trial
simulations. Sounds like you are over-estimating the variance. You might
want to look at a histogram of the ETAs for CL/F and look for departures
from normality.
Ken
Quoted reply history
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Nick Holford
Sent: Tuesday, April 22, 2008 4:04 PM
To: Ron Mathôt
Cc: [email protected]
Subject: Re: [NMusers] truncation & simulation
Ron,
When you truncate the simulated parameter distribution it can lead to a
major violation of the assumptions of maximum likelihood i.e. that all
random effects are normally distributed. This means that the likelihood
ratio test will have a larger Type 1 error than expected from using the
chi-2 distribution assumption. You should use a randomization test in order
to determine what change in OFV is needed in order to reject the null under
your desired hypothesis.
Nick
Ron Mathôt wrote:
Dear NONMEM users,
Currently I am working on the simulation of a bio-equavalence trial. For the
reference compound a population PK model has been derived on basis of data
from 100 patients. Values for between-and within-patient variability are
available for all PK parameters. The simulation comprises a randomized
cross-over study with 12 patients taking the test and reference compound.
Two-hunderd trials are simulated and summarized. During the simulations I
noticed that truncation of the simulated of PK parameters significantly
influences the power of the study to confirm bio-equivalence. For instance
truncation of simulated oral clearances of both compounds from a range of
1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to
decreased within- patient variability). Post-hoc estimates form the popPK
study indicated that clearance values of the reference compound are all
within the latter range of 5 to 30 L/hr. I expect that oral clearance of the
test compound will not differ more than 5% from the reference compound. In
my opinion simulation of trials with the smallest range will produce more
reliable estimates of the power to detect bio-equivalence.
I would greatly appreciate your comments on this subject.
Best regards,
Ron Mathôt
Department of Hospital Pharmacy and Clincal Pharmacology
Erasmus University Medical Center
Rotterdam
The Netherlands
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
[EMAIL PROTECTED] tel:+64(9)373-7599x86730 fax:+64(9)373-7090
www.health.auckland.ac.nz/pharmacology/staff/nholford
Dear Ron,
Your original dataset provided clearance estimates for 100 patients, which were
all within the range 5 to 30 L/h. When you simulated 4800 clearance values (=
2 cross-over observations/subject x 12 subject/trial x 200 trials), I would
expect to see a wider range of clearance values in the simulation dataset than
in the original analysis, simply because there are many more observations in
the simulation dataset.
However, arbitrarily truncating clearance to the range of 5 to 30 L/h may not
be wise. Consider the subject whose simulated clearance values are 30 L/h and
32 L/h, a 6.7% difference. If both of these values are truncated to 30 L/h,
then there is zero within-patient difference in clearance values (and zero
within-patient difference in AUC), and the truncation procedure both (1)
reduces the mean within-subject treatment difference and (2) reduces the
within-subject treatment variability. Either one of these effects would
arbitrarily increase the statistical power of attaining bioequivalence.
Statistical power for a bioequivalence test depends on the projected mean
within-subject treatment difference (say <=5%), the projected mean
within-subject variability (estimated in your original population PK analysis),
and the sample size. Instead of truncating clearance values, I think you
should examine the effects of larger sample sizes on the statistical power;
maybe run simulations with 16, 24, 36, and 48 subjects per study. In reality,
very few bioequivalence trials will be successful with only 12 subjects.
Good luck,
Steve
Steven Troy
Sr. Director,
Clinical Pharmacology and Pharmacokinetics
Shire Pharmaceuticals
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ron Mathôt
Sent: Tuesday, April 22, 2008 3:18 PM
To: [email protected]
Subject: [NMusers] truncation & simulation
Dear NONMEM users,
Currently I am working on the simulation of a bio-equavalence trial. For
the reference compound a population PK model has been derived on basis
of data from 100 patients. Values for between-and within-patient
variability are available for all PK parameters. The simulation
comprises a randomized cross-over study with 12 patients taking the
test and reference compound. Two-hunderd trials are simulated and
summarized. During the simulations I noticed that truncation of the
simulated of PK parameters significantly influences the power of the
study to confirm bio-equivalence. For instance truncation of simulated
oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30
L/hr doubled the number of positive trials (due to decreased within-
patient variability). Post-hoc estimates form the popPK study indicated
that clearance values of the reference compound are all within the
latter range of 5 to 30 L/hr. I expect that oral clearance of the test
compound will not differ more than 5% from the reference compound. In my
opinion simulation of trials with the smallest range will produce more
reliable estimates of the power to detect bio-equivalence.
I would greatly appreciate your comments on this subject.
Best regards,
Ron Mathôt
Department of Hospital Pharmacy and Clincal Pharmacology
Erasmus University Medical Center
Rotterdam
The Netherlands
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Please consider the environment before printing this e-mail
This email and any files transmitted with it are confidential and
may be legally privileged and are intended solely for the use of
the individual or entity to whom they are addressed. If you are
not the intended recipient please note that any disclosure,
distribution, or copying of this email is strictly prohibited and may
be unlawful. If received in error, please delete this email and any
attachments and confirm this to the sender.
www.shire.com
Dear Ron,
Do you have parameter covariances? or does your PK model have only diagonal
between-subject variability (OMEGA). Simulating from a full covariance model
can help eliminate non-physiologic parameter combinations.
For example, if CL and V are even somewhat correlated, simulating with a
diagonal variance matrix ignores this correlation.
The high and low CL (of your simulated subjects) may not be your "problem." It
could be the simulation of high CL (paired with unreasonably low Vol) or low CL
(paired with unreasonably high Vol)... for example.
I hope this helps,
Dave
Quoted reply history
On Tue, 22 Apr 2008, [ISO-8859-1] Ron Mathôt wrote:
> Dear NONMEM users,
>
> Currently I am working on the simulation of a bio-equavalence trial. For the
> reference compound a population PK model has been derived on basis of data
> from 100 patients. Values for between-and within-patient variability are
> available for all PK parameters. The simulation comprises a randomized
> cross-over study with 12 patients taking the test and reference compound.
> Two-hunderd trials are simulated and summarized. During the simulations I
> noticed that truncation of the simulated of PK parameters significantly
> influences the power of the study to confirm bio-equivalence. For instance
> truncation of simulated oral clearances of both compounds from a range of
> 1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to
> decreased within- patient variability). Post-hoc estimates form the popPK
> study indicated that clearance values of the reference compound are all
> within the latter range of 5 to 30 L/hr. I expect that oral clearance of the
> test compound will not differ more than 5% from the reference compound. In
> my opinion simulation of trials with the smallest range will produce more
> reliable estimates of the power to detect bio-equivalence.
>
> I would greatly appreciate your comments on this subject.
> Best regards,
>
> Ron Mathôt
>
> Department of Hospital Pharmacy and Clincal Pharmacology
> Erasmus University Medical Center
> Rotterdam
> The Netherlands
>
>
>
>
Steven,
While I agree that simulation of a larger number of subjects will result in
a wider range of clearance I have my doubts that that alone would be
sufficient to account for a 50-fold increase in range (5 to 30 vs 1 to 300).
Another contributing factor is that the post hoc estimates will have some
shrinkage because of empirical Bayes estimation (variance of post hoc
estimates of clearance will be smaller than the estimate of omega for
clearance). But again, I doubt that either of these explanations will fully
account for this 50-fold discrepancy. I still think Ron should look more
carefully at how the random effects are being modeled and further assess
whether his model can adequately describe the variability before using this
model in clinical trial simulations.
Ken
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Troy, Steven
Sent: Tuesday, April 22, 2008 5:00 PM
To: Ron Mathôt
Cc: [email protected]
Subject: RE: [NMusers] truncation & simulation
Dear Ron,
Your original dataset provided clearance estimates for 100 patients, which
were all within the range 5 to 30 L/h. When you simulated 4800 clearance
values (= 2 cross-over observations/subject x 12 subject/trial x 200
trials), I would expect to see a wider range of clearance values in the
simulation dataset than in the original analysis, simply because there are
many more observations in the simulation dataset.
However, arbitrarily truncating clearance to the range of 5 to 30 L/h may
not be wise. Consider the subject whose simulated clearance values are 30
L/h and 32 L/h, a 6.7% difference. If both of these values are truncated to
30 L/h, then there is zero within-patient difference in clearance values
(and zero within-patient difference in AUC), and the truncation procedure
both (1) reduces the mean within-subject treatment difference and (2)
reduces the within-subject treatment variability. Either one of these
effects would arbitrarily increase the statistical power of attaining
bioequivalence.
Statistical power for a bioequivalence test depends on the projected mean
within-subject treatment difference (say <=5%), the projected mean
within-subject variability (estimated in your original population PK
analysis), and the sample size. Instead of truncating clearance values, I
think you should examine the effects of larger sample sizes on the
statistical power; maybe run simulations with 16, 24, 36, and 48 subjects
per study. In reality, very few bioequivalence trials will be successful
with only 12 subjects.
Good luck,
Steve
Steven Troy
Sr. Director,
Clinical Pharmacology and Pharmacokinetics
Shire Pharmaceuticals
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Ron Mathôt
Sent: Tuesday, April 22, 2008 3:18 PM
To: [email protected]
Subject: [NMusers] truncation & simulation
Dear NONMEM users,
Currently I am working on the simulation of a bio-equavalence trial. For
the reference compound a population PK model has been derived on basis
of data from 100 patients. Values for between-and within-patient
variability are available for all PK parameters. The simulation
comprises a randomized cross-over study with 12 patients taking the
test and reference compound. Two-hunderd trials are simulated and
summarized. During the simulations I noticed that truncation of the
simulated of PK parameters significantly influences the power of the
study to confirm bio-equivalence. For instance truncation of simulated
oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30
L/hr doubled the number of positive trials (due to decreased within-
patient variability). Post-hoc estimates form the popPK study indicated
that clearance values of the reference compound are all within the
latter range of 5 to 30 L/hr. I expect that oral clearance of the test
compound will not differ more than 5% from the reference compound. In my
opinion simulation of trials with the smallest range will produce more
reliable estimates of the power to detect bio-equivalence.
I would greatly appreciate your comments on this subject.
Best regards,
Ron Mathôt
Department of Hospital Pharmacy and Clincal Pharmacology
Erasmus University Medical Center
Rotterdam
The Netherlands
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Please consider the environment before printing this e-mail
This email and any files transmitted with it are confidential and
may be legally privileged and are intended solely for the use of
the individual or entity to whom they are addressed. If you are
not the intended recipient please note that any disclosure,
distribution, or copying of this email is strictly prohibited and may
be unlawful. If received in error, please delete this email and any
attachments and confirm this to the sender.
www.shire.com
When you truncate your simulated parameters, you are actually changing
(narrowing) the distribution of the parameters and understandably increasing
the power in test.
Alan
Quoted reply history
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Ron Mathôt
Sent: Tuesday, April 22, 2008 3:18 PM
To: [email protected]
Subject: [NMusers] truncation & simulation
Dear NONMEM users,
Currently I am working on the simulation of a bio-equavalence trial. For
the reference compound a population PK model has been derived on basis
of data from 100 patients. Values for between-and within-patient
variability are available for all PK parameters. The simulation
comprises a randomized cross-over study with 12 patients taking the
test and reference compound. Two-hunderd trials are simulated and
summarized. During the simulations I noticed that truncation of the
simulated of PK parameters significantly influences the power of the
study to confirm bio-equivalence. For instance truncation of simulated
oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30
L/hr doubled the number of positive trials (due to decreased within-
patient variability). Post-hoc estimates form the popPK study indicated
that clearance values of the reference compound are all within the
latter range of 5 to 30 L/hr. I expect that oral clearance of the test
compound will not differ more than 5% from the reference compound. In my
opinion simulation of trials with the smallest range will produce more
reliable estimates of the power to detect bio-equivalence.
I would greatly appreciate your comments on this subject.
Best regards,
Ron Mathôt
Department of Hospital Pharmacy and Clincal Pharmacology
Erasmus University Medical Center
Rotterdam
The Netherlands
Dear all,
Thanks for the responses. Upon closer inspection of my controlstream I noticed that with the smaller range of 5 to 30 L/hr I also included a total covariance matrix. The latter proved to be the cause of the increased power as suggested by David. Further simulations (with covariance matrix) revealed that parameter distributions can be truncated from a 99.999 to a 95% interval without influencing the power of the study.
Ron
Ron Mathôt wrote:
> Dear NONMEM users,
>
> Currently I am working on the simulation of a bio-equavalence trial. For the reference compound a population PK model has been derived on basis of data from 100 patients. Values for between-and within-patient variability are available for all PK parameters. The simulation comprises a randomized cross-over study with 12 patients taking the test and reference compound. Two-hunderd trials are simulated and summarized. During the simulations I noticed that truncation of the simulated of PK parameters significantly influences the power of the study to confirm bio-equivalence. For instance truncation of simulated oral clearances of both compounds from a range of 1-300 L/hr to 5 - 30 L/hr doubled the number of positive trials (due to decreased within- patient variability). Post-hoc estimates form the popPK study indicated that clearance values of the reference compound are all within the latter range of 5 to 30 L/hr. I expect that oral clearance of the test compound will not differ more than 5% from the reference compound. In my opinion simulation of trials with the smallest range will produce more reliable estimates of the power to detect bio-equivalence.
>
> I would greatly appreciate your comments on this subject.
> Best regards,
>
> Ron Mathôt
>
> Department of Hospital Pharmacy and Clincal Pharmacology
> Erasmus University Medical Center
> Rotterdam
> The Netherlands