Fleiss multi-rater agreement
Estimate nominal agreement from category counts per subject with equal numbers of ratings.
Your entries stay in this page’s memory and are not submitted to us. Downloads are generated locally. Reloading or leaving the page clears your entries. Do not enter patient information.
Use 2–100 subjects and 2–10 categories. All cells are whole counts and every subject must have the same number of ratings, at least two.
The starting values are hypothetical. Results update as you edit. Use aggregate or synthetic inputs only.
Calculation results
- Mean observed agreement
- 0.58333333
- Expected agreement
- 0.34722222
- Fleiss kappa
- 0.36170213
How to interpret the output
Fleiss kappa compares mean pairwise observed agreement with agreement expected from pooled category proportions. It assumes a common rating structure and is not interchangeable with a two-rater Cohen kappa or an ordinal weighted statistic. Marginal prevalence affects interpretation; no confidence interval is returned.
Method referenceSource checked 2026-11-06. Read the reproducible calculator validation report.