Audit Treatment Overlap Before Trusting a Weighted Comparison

Connect overlap to the target population

A causal comparison requires support for the relevant treatment alternatives within the covariate patterns represented by the target. Begin by stating the population and treatment strategies. A comparison among all eligible people may require support that a comparison among a narrower overlap population does not.

Petersen and colleagues’ positivity paper discusses the consequences of limited data support and possible responses. Distinguish structural impossibility, such as a treatment never being available for a subgroup, from practical sparsity in a finite sample. Increasing model complexity cannot create observations for a structurally unavailable alternative.

Inspect support in several ways

Plot estimated propensity-score distributions by treatment and inspect clinically important covariate combinations directly. A single overlaid histogram can hide sparse regions or dependence on model specification. Record extreme estimated scores, unusually large weights and the number of observations contributing substantial information in each group.

Check whether observed lack of overlap reflects a coding error, eligibility mismatch or treatment definition that uses future information. Correct those problems before considering trimming. Keep the initial estimand visible when evaluating alternative weighting schemes or population restrictions.

Examine a hypothetical extreme score

Suppose a treated person has estimated treatment probability 0.01. Their unstabilized ATE weight is 1/0.01 = 100. A treated person with probability 0.50 has weight 2. The first record can therefore exert much greater influence, even though both represent one observed person.

Now suppose an entire clinically defined subgroup contains only treated people. A fitted logistic model may still output probabilities below one, but that numerical result does not demonstrate empirical support for untreated outcomes in the subgroup. Any estimate for that contrast depends heavily on extrapolation or additional assumptions.

Make a deliberate response

Possible responses include revising an unsupported target, collecting more relevant data, considering a justified overlap estimand or reporting that the original question is not adequately supported. Trimming or truncation changes the analysis and can change its bias properties; it is not a universal repair.

Prespecify plausible sensitivity settings where possible and report how many observations and how much weighted mass are affected. Recheck covariate balance and the population represented after each change. Save the original support diagnostics alongside the revised analysis, and describe residual extrapolation honestly. A stable numerical estimate should not be mistaken for evidence that the necessary treatment comparisons were observed.

Source verification: 6 November 2026. Examples are hypothetical.