Holm and Benjamini–Hochberg: A Practical Adjustment Workflow

Define the family before calculating

Create a list of the hypotheses that belong together, with a stable identifier for each one. Explain why they form a family: for example, several prespecified outcomes supporting the same scientific claim. Record the number of planned tests, including tests that might be inconvenient to report.

Do not select only small p-values for adjustment. A calculation applied to an incomplete list can give an apparently reassuring answer while missing the multiplicity problem. Use the multiplicity planning worksheet to document the family and decision rule before interpreting results.

Match the method to the objective

Holm controls the family-wise error rate: the probability of one or more false rejections within the family. Benjamini–Hochberg, commonly abbreviated BH, controls the false discovery rate under its applicable assumptions. FDR concerns the expected proportion of false rejections among rejections, with the proportion defined as zero when there are none; it is not the probability that a particular finding is false.

R's official p-value adjustment documentation explains these methods and their distinctions. Holm's control does not require independent tests. BH requires independence or appropriate positive dependence conditions; it should not be described as guaranteeing control under arbitrary dependence.

Check a fictional calculation

Take a complete four-test family with raw p-values 0.01, 0.03, 0.04, and 0.20. Holm-adjusted values are 0.04, 0.09, 0.09, and 0.20. BH-adjusted values are 0.04, approximately 0.05333, approximately 0.05333, and 0.20.

At a prespecified 0.05 threshold, only the first test meets either adjusted rule in this invented example. Try the multiple-testing adjuster, then verify that its results remain attached to the correct hypothesis identifiers when inputs are reordered.

Keep interpretation attached to the plan

The procedures sort values internally, so tied or reordered inputs deserve a careful output check. Adjustment cannot repair biased analyses, unreliable measurements, selective outcome reporting, or a family defined after seeing the results. It also does not supply an effect size or confidence interval.

Report the raw estimates and their uncertainty alongside the adjustment method, family definition, and adjusted p-values. If the analysis is exploratory, say so. Avoid changing from Holm to BH merely because one produces more discoveries; the choice changes the error-control objective, not just the formatting of a results table.

Source checked on 30 October 2026; numerical example independently recalculated.