Pilot questionnaire burden with real task observations

Define burden beyond the item count

A short questionnaire can still be difficult if it asks for information respondents cannot retrieve, uses unclear concepts or has an awkward interface. Begin by listing the tasks required: reading, recalling, looking up records, choosing a response and navigating the form. Include accessibility and language needs relevant to the intended population.

The US Census questionnaire-testing guidance describes complementary pretesting approaches, including cognitive and usability methods. A burden pilot should reveal where effort is spent and why, rather than only produce one average completion time.

Observe the intended workflow

Test the actual administration mode and realistic sequence, including consent, instructions and branching. Ask participants to explain how they understood selected questions and found their answers. Record interruptions, assistance, backtracking, abandoned sections and technical problems separately from active completion time.

Choose participants who can expose relevant differences in experience, literacy, device use or record access. A convenience test among colleagues may identify broken controls but does not establish that the intended respondents can answer the questions as designed. Keep pilot feedback confidential and use synthetic examples in public documentation.

Hypothetical example

A fictional survey has 15 questions and a median completion time of eight minutes, but several respondents spend most of that time searching for an exact date they do not know. Removing unrelated short questions would not address the main burden. The team considers whether the exact date is essential, whether records can supply it and whether a justified less precise response option is acceptable.

Any change must preserve the intended measurement. Shortening a validated instrument by deleting inconvenient items is not automatically a valid solution.

Turn observations into revisions

Create an issue log containing the task, observed difficulty, likely cause, proposed revision and evidence needed to confirm improvement. Distinguish wording problems from navigation problems and genuinely demanding information requests. Retest the affected workflow after revisions, including any changed skip logic.

Report the pilot population, testing methods and resulting changes without claiming comprehensive validation from a small usability exercise. Use the collection pilot worksheet to record operational findings and the skip-logic truth table to test branches. The release decision should reflect whether respondents can complete the intended task accurately and with acceptable effort, not merely whether the form looks brief on a screen.

Source verification: 6 November 2026. Examples are hypothetical.