Distinguish Leverage, Residual Size, and Influence

Separate three different questions

A large residual means an observation differs from its fitted outcome. High leverage means its predictor values are unusual relative to the model’s design. Influence concerns how much the fitted model or a target estimate changes because of that observation. One point can have one, two or all three features.

The statsmodels regression-diagnostics example includes influence measures and related checks. Use these measures to prioritize investigation, while remembering that thresholds depend on model structure and are not universal scientific deletion rules.

Preserve the observation’s context

Inspect the source record, units, eligibility and measurement process. A high-leverage observation may be a valid participant from an underrepresented range. Removing it can narrow the population to which the analysis applies. Conversely, a data-entry error deserves correction even if its influence measure is small.

Consider influence on the specific estimand, not only overall fit. An observation can have little effect on average predictions but substantially alter an interaction coefficient or a contrast near the edge of the data.

Compare a hypothetical sensitivity check

Suppose the fitted treatment coefficient is 2.0 units with all 100 observations. Omitting one flagged observation changes it to 1.9, while omitting another changes it to 0.8. The absolute changes are 0.1 and 1.2 units. Those changes describe sensitivity; they do not establish that the second observation is erroneous.

Check whether the second record is valid, whether it lies in a sparsely represented predictor region, and whether the model is too rigid for that region. If it is a confirmed measurement error, correct it using source evidence. If valid, retain it in the primary analysis unless a prespecified eligibility rule says otherwise, and explain the sensitivity where important.

Avoid an exclusion cascade

Repeatedly removing the most influential point until the result stabilizes can change both the estimand and error properties of the analysis. Keep a complete log of investigated observations and reasons for any exclusion or correction.

Report robust or alternative model analyses when scientifically justified, naming what they estimate and what assumptions they change. The final diagnostic package should distinguish data corrections, eligibility exclusions and sensitivity analyses. A model becomes more interpretable when influential observations are understood in context, not merely absent from the final dataset.

Source verification: 6 November 2026. Examples are hypothetical.