Explain How Case Mix Changes External Model Performance

Define the population being represented

External validation asks how a fixed prediction rule performs in data that were not used to develop it. Describe the setting, eligibility, referral pathway and outcome horizon before presenting performance metrics. A new hospital name alone does not establish how different the validation population is from the original one.

The BMJ’s external-validation guidance emphasizes the suitability and target population of validation data. Restricted eligibility can leave important groups unevaluated. State those limits rather than treating performance in one selected cohort as evidence for every intended user.

Build a comparability table

Compare predictor definitions, units, missingness, distributions and the range of predicted risks. Describe outcome ascertainment and follow-up completeness. Include meaningful clinical categories and measurement procedures, not only p-values comparing cohorts; large samples can make trivial differences statistically significant.

Separate case mix from changes in predictor effects, baseline risk and data quality. A model’s discrimination can change when the range of risk profiles changes, even if the underlying predictor relationships remain similar. Calibration and discrimination therefore need separate evaluation and explanation.

Consider a hypothetical restriction

Imagine a model developed across a broad age and severity range, then evaluated only among moderately ill people aged 50–60. The validation cohort offers a narrower spectrum of predictors and outcomes. A lower C statistic in that restricted group would not, by itself, prove that the model coefficients were implemented incorrectly.

Conversely, combining several sharply different populations can make ranking appear strong while hiding poor calibration within each setting. The practical response is to inspect the relevant populations and risk ranges, not to choose the broadest sample simply because it produces the largest discrimination statistic.

Report what was and was not tested

Evaluate calibration curves and clinically relevant risk ranges with appropriate uncertainty. Prespecify subgroup assessments and acknowledge sparse groups rather than presenting unstable estimates as definitive failures or successes. Preserve the original prediction formula and document any missing predictor substitutions.

If recalibration is explored, report the original validation separately from the updated model’s assessment. Describe which differences could plausibly explain the results and which remain unresolved. A useful conclusion names the population for which evidence was obtained and the groups or settings that still need evaluation. External validation is a study of transportability under observed conditions, not a permanent certificate attached to a model’s name.

Source verification: 6 November 2026. Examples are hypothetical.