Fwd: Should we generate VPCs with or without uncertainty?

Hi all, Creation of VPCs is a way to assess if simulated data generated by the model is compatible with observed data. VPCs are usually based on parameter point estimates of the model. Sometimes parameter uncertainty is also accounted for in the generation of VPCs (PPCs) where each simulated replicate of the data set is based on a new set of parameter values representing the uncertainty of the estimates (e.g. based on a bootstrap). I wonder if inclusion of uncertainty in this way is really appropriate or if it just makes the confidence intervals wider and hence easier to qualify the model. Is it possible based on such an approach, that a model might look good, when in fact no likely combination of parameter values (based on parameter uncertainty) would generate data that are compatible with the observations? To illustrate my question: I could generate 100 sets of parameters reflecting parameter uncertainty (e.g. from a bootstrap). Based on each set of parameters I could then generate a separate VPC (e.g. showing median, 5 and 95% percentile) to see if any of the parameter sets are compatible with data. I would then have 100 VPCs, each based on a separate set of parameter values reflecting the parameter correlations and uncertainty. If the VPC based on point estimates looks bad, I would (generally) expect that the other VPCs would be worse (they all have lower likelihood), so that we have 101 VPCs that does not look good. Some might over predict and some underpredict, some might describe parts of the relation better than the VPC based on the point estimates. By putting the VPCs together from all parameter vectors, the CI becomes wider, and perhaps now includes the observed data. So based on a set of 100 parameter vectors which individually are not compatible with the observed data I have now generated a VPC (PPC) where the confidence interval actually includes the observed metric (e.g median). It seems to me that based on such an approach it is possible that a model might look good, when in fact no likely individual set of parameter values would generate data that are compatible with the observations. Simulation based on parameter uncertainty is useful when we want to make inference, but I am unsure of its use for model qualification. In any case it is confusing that we some times simulate based on point estimates and sometimes based on parameter uncertainty without any particular rationale as far as I understand. Would be interested if someone could shed some light on the inclusion of uncertainty in simulations for model qualification (VPCs). Best regards, Matts Kagedal Pharmacometrician, Genentech