Statistical Methods

Handle zero-weight logarithmic terms explicitly

Define the task

Logarithmic scores and information summaries often contain terms such as x times log(y). Specify the intended domain before replacing missing or invalid values. A zero weight and a missing observation are different data states. An implementation should preserve that distinction in its input validation and output.

Carry out the check

The function xlogy defines the zero-weight case as zero, avoiding an undefined intermediate result when directly multiplying zero by log(0). For positive weights and zero y, the logarithmic contribution can remain infinite. Do not add a small constant to every value without documenting the changed calculation. Check the package's handling of NaN inputs separately.

Worked hypothetical example

For a hypothetical categorical distribution with probabilities (0,0.5,0.5), the entropy contribution of the zero category is defined by the limit of −p log(p), which approaches zero. The total entropy is log(2), about 0.693147 natural-log units. In a prediction score, assigning probability zero to an event that actually occurs has a different meaning and can give infinite loss. Keep these cases separate in numerical tests.

Method checked 14 November 2026 against SciPy documentation: scipy.special.xlogy.

Related reading

Reading path: Analyze data. Guide 142 of 142. Choose any guide to suit your project.

Statistical Methods

Handle delayed entry in a survival analysis

Align the time origin and entry time so participants join the risk set only when they become observable.

Statistical Methods

Review Weight Trimming as a Bias–Precision Tradeoff

Investigate influential survey weights and document a justified trimming sensitivity analysis.

Browse this reading pathTools for this stageGuides and downloads for this stage