Pilot Eligibility Criteria Before Full-Text Screening

Translate the question into decisions

Eligibility criteria must help two reviewers reach a defensible decision from an actual report. A phrase such as “adult rehabilitation studies” leaves important questions unanswered: mixed-age samples, inpatient versus outpatient care, intervention components, and eligible designs all need operational definitions.

Cochrane's chapter on review scope and eligibility places these decisions within the review's planned population, intervention, comparison, and study-design scope. Write the criteria before judging whether a study's result is attractive or convenient.

Assemble a purposeful pilot set

Choose a small set of reports that tests the edges of your definitions. Include clearly eligible and ineligible reports, plus plausible borderline cases: mixed populations, unusual comparators, unclear designs, or companion publications. This is a practical training set, not a statistical estimate of screening accuracy.

Record a stable report identifier, each reviewer's initial decision, the exact criterion used, and the supporting passage or page. Keep “unclear” separate from “exclude” when essential information is missing. Decide in advance how reviewers will seek clarification and resolve disagreements.

Diagnose disagreements before voting

Have reviewers apply the draft rules independently, then compare the reason for each disagreement. Ask whether the problem is ambiguous wording, missing information, inconsistent interpretation, or a genuine scope decision. Revise the rule with an example that clarifies how to handle the next similar report.

For instance, a fictional report might enroll participants aged 16 to 70 without an adult-only result. An “adult population” criterion needs an explicit approach to this situation. Do not invent an age proportion cutoff after noticing which studies would be retained; document and justify any protocol amendment.

Separate eligibility from data availability

An otherwise eligible study may omit usable numerical results for a relevant outcome. Distinguish “outcome not measured” from “outcome measured but not reported” and “report unclear.” Cochrane cautions against excluding studies merely because outcome results are unavailable. Preserve the study and document what can be established under the protocol's rules.

Link companion reports to the same study before counting included studies. A conference abstract, registry entry, and journal article may supply complementary information rather than three separate studies.

Finish the pilot by freezing a dated criteria version and a short decision log. If the rules change later, identify which previously screened records need reassessment. The useful output is a repeatable set of decisions, not a ceremonial percentage agreement with unresolved differences underneath.

Source checked on 30 October 2026; examples are fictional.