Manuscript Writing
Common Statistical Errors in Medical Manuscripts and How to Avoid Them
Updated
Statistical review should connect the research question, study design, analysis, and interpretation. A checklist can expose inconsistencies, but it cannot replace review of the data and methods. The following checks address common sources of misleading conclusions.
1. Interpret p-values without reversing the conditional probability
A p-value describes the probability, under the specified null model and its assumptions, of a test statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true, that the findings occurred by chance, or that a result will replicate.
Report effect estimates and uncertainty alongside p-values. Statistical significance does not establish clinical importance, and crossing a threshold should not be the sole basis for a conclusion. Explain which analyses were planned and disclose relevant exploratory analyses rather than selecting only favorable results. American Statistical Association statement
Check that estimates, confidence intervals, and p-values refer to the same model, contrast, and method. A mismatch may reflect a transcription error or different inferential procedures; explain genuine differences rather than presenting inconsistent numbers.
2. Check assumptions relevant to the chosen analysis
Describe the analysis population, dependence between observations, outcome scale, and model specification. Use diagnostics to investigate potential problems; a single diagnostic p-value does not certify that a model is appropriate.
Do not switch automatically to Mann–Whitney simply because an outcome looks skewed. Changing the test can change the question being answered: Mann–Whitney can detect distributional differences beyond a shift in medians. State the intended interpretation and justify it. Hart, BMJ
If a diagnostic leads to a changed analysis, report the reason and distinguish the planned method from the revised method. Where the change matters, show how conclusions depend on it.
3. Explain missing-data assumptions
Complete-case analysis can discard information and introduce bias, but it is not valid only under missing completely at random. Validity depends on the missingness process, the outcome, and the analysis. For example, some regression analyses with missing outcomes can remain valid under an appropriate missing-at-random assumption conditional on included covariates.
Multiple imputation is one option, not an automatic remedy. Its performance depends on the imputation model and assumptions; ordinary MAR-based imputation does not resolve plausible missing-not-at-random mechanisms. Report missingness, model variables, software, number of imputations, pooling procedure, and sensitivity analyses. Select enough imputations for stable inference rather than treating 5–10 as a universal default. Sterne and colleagues, BMJ
4. Define the testing family and error rate
For 20 independent tests with all null hypotheses true and each test conducted at α = 0.05, the probability of at least one false rejection is 1 − 0.95²⁰ ≈ 64.2%. That exact calculation does not apply to arbitrary dependent tests.
Trials can have multiple primary or co-primary endpoints. The appropriate testing strategy depends on whether success requires all endpoints or any endpoint and on the intended claims. Pre-specify the endpoint structure and multiplicity strategy. Family-wise error control and false-discovery-rate control answer different questions; neither should be selected merely because it produces more favorable findings. Secondary analyses are not necessarily exploratory if they were prospectively included in a valid confirmatory strategy. FDA guidance on multiple endpoints
5. Match causal language to the design and assumptions
An adjusted association alone does not establish causation. However, observational data can support causal inference under explicitly stated identification assumptions and suitable design and analysis. These include consistency, exchangeability, and positivity; they can be demanding and cannot generally be established by a regression output.
Define the causal question if there is one, explain the assumptions and likely threats, and discuss residual confounding and selection bias. Otherwise, describe the result as an association. Hernán and Robins, Causal Inference: What If, chapter 3
Before submission
Reconcile tables with the analysis output, document deviations from the analysis plan, and ensure the conclusions reflect the estimates and limitations. Keep the code and relevant analysis records available under the project's data-handling agreement.
For a methods review, discuss your manuscript with StatMedAnalysis.