Prediction models need reliable performance for diagnosis, prognosis, and treatment planning.Changes in case-mix can impact model performance.Discrimination and calibration respond differently depending on the causal direction of the prediction task.Understanding the causal structure of the prediction task is important for developing and evaluating prediction models in different clinical settings.