Audit Standardized Differences for Covariate Balance

Define the balance question

List the covariates whose distributions you want to compare and the population represented by each group. Record whether the summaries describe the original sample, a matched sample, or weighted observations. Balance assessment concerns those observed covariates; it does not estimate a treatment effect or establish that unmeasured confounding is absent.

The author-maintained cobalt balance documentation describes several standardization choices and distinguishes raw from standardized differences. A number called a standardized difference is not fully specified until its denominator is documented.

Keep the standardization convention explicit

For continuous summaries, the demonstration tool divides the first mean minus the second by the square root of the average of their two variances. This equal-group variance convention differs from the sample-size-weighted pooled SD used in some effect-size calculations. No Hedges small-sample correction is applied.

For a binary covariate, it divides the difference in proportions by the square root of the average of p1(1−p1) and p2(1−p2). Preserve the original proportions alongside that value: standardization can make a simple percentage-point difference harder to recognize.

Recalculate fictional summaries

Suppose the first group's mean age is 50 and the second's is 55, with SD 10 in each group. The standardized difference is (50−55)/10 = −0.50. The sign identifies the order of subtraction; its absolute value describes the size under this convention.

For a fictional binary covariate present in 20% and 40% of the groups, the denominator is sqrt((0.16+0.24)/2), approximately 0.447214. The standardized difference is approximately −0.447214, while the raw difference is −20 percentage points. Use the standardized-balance tool to verify these examples.

Review more than the mean difference

The balance-diagnostic worksheet records variable definitions, populations, summaries, and standardization rules. Inspect spread and distribution shape as well as mean differences. Equal means can coexist with different tails or subgroup mixtures that matter for the analysis.

When comparing balance before and after adjustment, decide how the denominator will be held consistent. Replacing it with a newly calculated post-adjustment spread can change the diagnostic even without the intended change in the numerator. The simple summary-input tool does not generate weights, perform matching, or retain a baseline denominator automatically.

Do not treat a conventional numerical cutoff as proof of exchangeability or use a nonsignificant baseline test as evidence of balance. Document remaining discrepancies and their implications for the planned analysis. Finish with the raw summaries, chosen diagnostic, and a concise account of what the assessment can and cannot establish about the observed comparison.

Source checked on 4 November 2026; fictional calculations independently recomputed.