RE: Using MCP-MOD in dose finding for Phase 3

Dear Nele, Dear all, Below in red in Nele's e-mail, you will find the input of Bjoern Bornkamp, a statistician from the Novartis Stats/Methods group. I forwarded your mail to him. Bjoern was involved in the qualification discussion with EMA together with Jose Pinheiro and Frank Bretz. He is one of the implementers of the MCP-Mod methodology within Novartis, and applies it routinely in Phase 2 studies. I am sure that Bjoern' answers will help. Bye Jean-Louis Steimer +++++++++++++++++++++++ Dear all, I am writing to you as we are currently discussing the implementation of the MCP-MOD approach for dose finding based on Phase 2B results and would like to hear your opinion on this approach. Original MCP-Mod is not intended to be used in Ph III, special adaptations are necessary (closed testing). It would be good to get feedback from both statisticians and classical modelers. I have thought about the approach, and have a few problems about seeing the advantage of the approach over complete population-PK/PD modeling. These are two different approaches that complement each other. MCP-Mod is not intended to replace population-PK/PD modeling (the idea is to replace ANOVA-type models). I can see benefits to do both a simple cross-sectional dose-response analysis (like MCP-Mod) and a complete dose-exposure-response characterization. If results are consistent between both approaches one would have more confidence overall in the analysis results than from either analysis alone. If results are not consistent one needs to dig a bit "deeper", but this is also useful information. >From what I understood, I can see the following issues: MCP-MOD · Only uses trial endpoints, i.e. it ignores the time course of the treatment effect. I have a problem with this because there might be noise in the endpoint (e.g. if the effect has reached a plateau), which might potentially lead to the selection of the wrong model structure. Including the time-course like in PKPD modeling approaches would detect that the deviation is just noise, and thus probably be able to identify the right model structure despite this. MCP-Mod can handle longitudinal data, see Pinheiro et al. (2014), Stat Med. 33,1646-61 for one example, which is also available in the DoseFinding R package. · Uses dose-response models instead of exposure-response models Correct. Again, MCP-Mod is not intended to replace population-PK/PD modeling. We have started thinking how to extend the key ideas of MCP-Mod to exposure-response models and encourage the community to look into this. · Pre-specifies the model structure. While I understand that for pivotal trials prespecification is crucial, I would assume that Phase 2 is performed to allow exploration of the data to come up with the best model given the data we have. What happens if the true model is not part of the tested ones? What if we have new physiological insights that tell us about the model structure after we have seen the data? Do we then ignore what we know and fit all bad models, and if none gives a good description we do model averaging of bad models? Excellent questions. Candidate models for MCP-Mod should always be selected based on entire teams input and operating characteristics should be evaluated upfront. More specifically, our experience shows that MCP-Mod is relatively robust if the true model is not part of the tested ones, see for example Pinheiro et al. (2006), J. Biopharm. Statist. 16,639-656. This is also something that can be evaluated to some extend upfront (at the design stage) by simulations. Among other things one advantage of pre-specification is that it makes the modelling more transparent/credible for externals (e.g. health authorities), if one specifies before seeing the data what will be done. But of course there is a trade-off: Not sure if it is possible to pre-specify a full population PK/PD analysis. · If we include a model with many parameters in the prespecification and only have a few dose strength, wouldn't the model with more parameters be more likely to give a good fit (e.g. when comparing Emax to logistic), with the consequence that a wrong dose might be selected? Not sure whether I fully understand this question. Of course the model-selection/averaging step of MCP-Mod would take into account the model complexity by using AIC/BIC (not only looking at model fit). Again, operating characteristics need to be evaluated in advance, which include precision of target dose estimation and also possible convergence problems if the number of parameters is to larger. Colleagues from statistics recommend to cover all potential models with different shapes in the candidate set to avoid potential bias in dose selection, but they argue that post-hoc model fitting leads to data-dredging and over-fitting, does not account for model uncertainty and gives overly-optimistic results. I am wondering however what the difference in the approach is if anyway ALL potential models are considered (which can lead to overfitting as well)? There is a penalty for using many models in MCP-Mod: In the MCP step the multiplicity adjustment would get higher if there are more models included (in particular if they are very different). In the Mod step the variance of the dose-response curve would increase with an increased number of models, so there one faces the usual variance/bias trade-off. Might a good solution be to combine PKPD modeling with MCP-Mod? Yes, see above Your opinion will be highly appreciated, and I am looking forward to receiving comments both in favour and against the approach :-) Best Nele