RE: order of covariate inclusion -> avoiding stepwise approaches -> abandoning exploratory analysis?
From: tgordi@buffalo.edu
Subject: RE: [NMusers] order of covariate inclusion -> avoiding stepwise approaches -> abandoning exploratory analysis?
Date: 9/29/2003 11:00 AM
Hi!
I am not as much experienced with NONMEM (or statistics) as many of the contributors
but I guess there are a couple of simple rules one can follow with regard to inclusion
of covariates in any model. The first is, of course, just consider those covariates that
are of interest. What is of interest might differ from case to case. If it is a exploratory
study, one might want to look at many parameters. If one want a more "practical" model, weight
and gender might be enough (just examples). A lot of information can be collected in a clinical
study but not all of it will be or should be tested. The second simple rule is that the more
tests one does, the higher is the risk of finding something that is not there (as Ken mentions
below). Although there probably are several methods to deal with the situation, one simple
approach would be to decrease the level of significance, let's say use 0.001 instead of 0.05,
for a covariate to be deemed influential. What value to choose will be dependent on the
number of tests you do, although there are some variations of the rule, which are less
conservative. The negative side is, of course, that there will be more difficult to show
any significance. However, this might function as a driving force to choose the best
candidates only. I understand that this is not the answer to all of the problems but I would
say that following these rules helps one to be less "wrong" in the analysis.
This discussion also brings me to another issue I have been thinking about. If we agree
that conducting several tests lead to inflated TYPE I error, what are the consequences
when several different structural models are tried in our routine PK/PD modeling? It is
a common practice to accept a drop of 3.8 in the OFV (which is related to a significance
level of 0.05) as a "proof" of the superiority of a model over another. My point is not
whether using OFV is optimal. The question is whether one should be more restrictive
and use 0.01 or even less.
Toufigh Gordi