Prespecify Confidence Levels Before Reading Results

Separate the level from the method

A confidence level and an interval construction are different choices. Specifying 95% does not tell an analyst whether to use a normal approximation, a t interval, a score method, or another procedure. Record both, together with the parameter, analysis scale, and study design.

The NIST explanation of confidence limits connects the confidence level to the repeated-sampling behavior of the procedure. It does not assign a probability that a fixed parameter lies in the particular frequentist interval already calculated.

Make the decision before inspecting significance

Write the intended level, whether the interval is one-sided or two-sided, and its relationship to the primary analysis in the analysis plan. If a decision boundary or non-inferiority margin matters, define it separately and justify it scientifically. Do not choose a narrower interval after noticing that the planned interval includes an inconvenient value.

The ICH E9 guidance describes detailed analysis planning and documentation of decisions. Preserve the timing and rationale for any change rather than presenting a post-result choice as prespecified.

Inspect a fictional width comparison

Suppose a fictional estimate is 2 with known or adequately estimated SE 1, and a normal approximation is appropriate. A two-sided 90% interval is approximately 0.3551 to 3.6449, using critical value 1.644854. A 95% interval is approximately 0.0400 to 3.9600, using 1.959964.

At 99%, the interval is approximately −0.5758 to 4.5758, using 2.575829. The estimate and SE have not changed; the interval width has. These examples show why choosing the level after examining whether zero is excluded can misrepresent the analysis.

Keep multiple intervals and sidedness explicit

Several separate 95% intervals do not automatically provide 95% simultaneous coverage for an entire family of parameters. If the study needs simultaneous inference or a testing hierarchy, specify the relevant procedure rather than assuming a familiar marginal level solves the problem.

A one-sided bound also answers a different question from a two-sided interval. Do not exchange them without explaining the decision and its consequences. Where a tool fixes the level, such as the Wilson-proportion tool, retain that limitation when using it for illustration rather than implying it implements every analysis plan.

Use the interval planning worksheet for binary examples and extend the same documentation principles to other parameters. Record rounding only as a display choice; calculations should retain adequate precision.

Finish with an output specification naming the estimate, method, level, sidedness, scale, and any multiplicity adjustment. A reader should be able to distinguish a change in evidence from a change in the rule used to display uncertainty.

Sources checked on 3 November 2026; fictional normal intervals independently recalculated.