Re: Cross-validation script in NM
hi Pieter,
It is not exactly clear from the NONMEM manuals what information is
included in the MSFO file, or the exact mechanism that is used to
re-implement the model, so it's hard to say if this should really work the
way you expect it to. To my knowledge (and the manual), model specification
files are intended for restarting the estimation process, and not really
for analyses where you use a different dataset in the second estimation (or
simulation) step. For an analysis such as yours, or in fact any simulation
or re-estimation analysis, I would always do the data mangling outside of
NONMEM, e.g. in R or Perl. That way you will have full control over what is
going on, and you'll use NONMEM only for what it is good at (i.e.
estimation).
hope this helps,
Ron
----------------------------------------------
Ron Keizer, PharmD PhD
Dept. of Bioengineering & Therapeutic Sciences
University of California San Francisco (UCSF)
----------------------------------------------
Quoted reply history
On Fri, Dec 5, 2014 at 11:03 AM, Pieter Colin <[email protected]> wrote:
> Dear Kajsa and Dennis
>
>
>
> Thank you for your thoughts on this.
>
> I know of (and have used several times in the past) the mentioned
> functionalities in PsN and PLT-tools.
>
> However, due to the specific nature of my problem, I’m afraid these will
> not work for me.
>
>
>
> Allow me to further clarify my problem. (For clarity, I’ve included a
> piece of my control stream at the bottom of this message.)
>
> As Dennis pointed out, I’m fitting a training group and use the final
> parameter estimates in a subsequent run to predict the plasmaconcentrations
> of the validation group.
>
> I failed to clarify this in my previous message, but I’m predicting the
> plasmaconcentrations for the validation group according to a TDM setting.
>
> This means that for the validation group MAXEVAL=0 and only the first
> through sample per ID is included in the dataset as an observation
> event(EVID=0 and MDV=0).
>
> It goes without saying that the objective is to accurately predict the
> other plasmaconcentrations (EVID=2 and MDV=1) for the IDs in the validation
> group.
>
>
>
> Now to get to the problem. I tried this approach with two separate control
> streams and it works.
>
> I.e. plasmaconcentrations are predicted for the validation group based on
> the post-hoc corrected final parameter estimates of the training group.
>
>
>
> However, when I combine these in a single control stream (as shown below)
> the time-varying covariates are not taken into account for the validation
> group.
>
> More specifically, the following statement (under $PK) is not evaluated
> for the IDs in the validation group (statement used to switch on/off an
> additional CL due to hemodialysis).
>
>
>
> CL_DIA = 0
>
> IF(DIALYSIS.EQ.1) CL_DIA = THETA(6)
>
>
>
> IND=0
>
> IF(IND_DIA.EQ.1) IND=1
>
>
>
> This causes the hemodialysis moments to be ignored by NM in the validation
> group when using the control stream as shown below.
>
> Since it worked for me using separate control streams, it seems that the
> problem is associated with the use of MSFO=… and $MSFI in the training and
> validation set, respectively.
>
> Do any of you have a specific solution for this problem or could shed some
> light on specific behavior of the $MSFI option in NM which might be causing
> this?
>
>
>
> Kind regards,
>
>
>
> Pieter Colin
>
>
>
>
>
> $PROBLEM No covariates
>
> ;; 1. Based on:
>
> ;; COMMENT:
>
>
>
>
> ;---------------------------------------------------------------------------------
>
> ;----------------------- FIT XVAL
> --------------------------------------------
>
>
> ;---------------------------------------------------------------------------------
>
>
>
> $INPUT ID TIME DV CMT AMT RATE EVID MDV UVOL EXTRA IND_DIA OCC
>
> DIALYSIS ANALYSIS BV MISSING AGE WGT HGT BMI BSA SOFA M1F2 GFR
> XVAL
>
>
>
> $DATA RawdataCFP_cov_ext.csv
>
> IGNORE=@ IGNORE(MISSING.EQ.1) ;Exclude missing
> values
>
> IGNORE(CMT.GT.3) ;Exclude CSF sample
>
> IGNORE(XVAL.EQ.1) REWIND
>
>
>
> $SUBROUTINE ADVAN13 TOL=12
>
>
>
> $MODEL COMP(CENTRAL,DEFOBS,DEFDOSE) COMP(PERIPH)
>
> COMP(URINE,INITIALOFF)
>
>
>
> $PK
>
> ;------------- Calculation of Time After Dose ------------
>
>
>
> IF (EVID.EQ.1.OR.EVID.EQ.4) THEN
>
> TDOS=TIME
>
> TAD=0.0
>
> ENDIF
>
> IF (EVID.NE.1.AND.EVID.NE.4) TAD=TIME-TDOS
>
>
>
> TVCLOTHER =THETA(1)
>
> CLOTHER =TVCLOTHER*EXP(ETA(4))
>
>
>
> TVCL = THETA(2)
>
> CL = TVCL*EXP(ETA(1))
>
>
>
> TVV1 = THETA(3)
>
> V1 =TVV1*EXP(ETA(2))
>
>
>
> TVV2 =THETA(4)
>
> V2 =TVV2*EXP(ETA(3))
>
>
>
> TVQ =THETA(5)
>
> Q =TVQ
>
>
>
> ;------------- Dialysis submodel ------------------------
>
> CL_DIA = 0
>
> IF(DIALYSIS.EQ.1) CL_DIA = THETA(6)
>
>
>
> IND=0
>
> IF(IND_DIA.EQ.1) IND=1
>
>
>
> S1=V1
>
> S3=UVOL
>
>
>
> K10=CLOTHER/V1
>
> K12=Q/V1
>
> K21=Q/V2
>
> K13=CL/V1
>
> K11=CL_DIA/V1
>
>
>
> $DES
>
> DADT(1)=-K12*A(1)+K21*A(2)-K10*A(1)-K13*A(1)-K11*A(1)*IND
>
> DADT(2)=K12*A(1)-K21*A(2)
>
> DADT(3)=K13*A(1)
>
>
>
> $ERROR
>
> IPRED = 1E-3
>
> IF(F.GT.0) IPRED=F
>
>
>
> Y = IPRED*(1+EPS(1))
>
> IRES = DV-IPRED
>
> IWRES = IRES/(IPRED*SQRT(SIGMA(1,1)))
>
>
>
> IF(CMT.EQ.3) THEN
>
> Y = IPRED*(1+EPS(2))
>
> IRES = DV-IPRED
>
> IWRES = IRES/SQRT(IPRED*IPRED*SIGMA(2,2))
>
> ENDIF
>
>
>
> $THETA
>
> (1E-9,1.097450) ; CLOTHER; L/h
>
> (1E-9,2.124530) ; CL; L/h
>
> (1E-9,8.640870) ; V1; L
>
> (1E-9,18.58180) ; V2; L
>
> (1E-9,34.13580) ; Q; L/h
>
> (1E-9,4.046690) ; CL_DIA; L/h
>
>
>
> $OMEGA
>
> 1.265890 ; IIV_CL
>
> 0.387112 ; IIV_V1
>
> 0.186287 ; IIV_V2
>
> 0.371892 ; IIV_CLOTHER
>
>
>
> $SIGMA
>
> 0.090199 ; Proportional plasma
>
> 0.106711 ; Proportional urine
>
>
>
> $ESTIMATION SIG=2 MAX=9999 METHOD=1 SORT INTERACTION POSTHOC PRINT=1
>
> MSFO=run61.msf
>
>
>
>
> ;---------------------------------------------------------------------------------
>
> ;----------------------- POST HOC
> --------------------------------------------
>
>
> ;---------------------------------------------------------------------------------
>
>
>
> $PROBLEM PREDICT XVAL1
>
>
>
> $INPUT ID TIME DV CP CMT AMT RATE EVID MDV UVOL EXTRA IND_DIA OCC
>
> DIALYSIS ANALYSIS BV MISSING AGE WGT HGT BMI BSA SOFA M1F2 GFR
> TDM XVAL
>
>
>
> $DATA RawdataCFP_xval_ext.csv
>
> IGNORE=@
>
> IGNORE(MISSING.EQ.1) ;Exclude missing
> values
>
> IGNORE(CMT.GT.3) ;Exclude CSF sample
>
> IGNORE(XVAL.NE.1) REWIND
>
>
>
> $MSFI run61.msf
>
>
>
> $ESTIMATION SIG=2 MAX=0 METHOD=1 SORT INTERACTION POSTHOC PRINT=1
>
>
>
> …
>
>
>