Check CSV Delimiters and Quoting Before Counting Records

Inspect the format before opening it

A CSV filename does not guarantee comma delimiters or one physical line per observation. Exporters can use semicolons, quoted fields and embedded line breaks. Opening the file with an assumed separator may silently shift values into the wrong columns. Preserve the original file and record its exporter, encoding and expected field names before import.

The Python CSV documentation describes dialects and quoting rules. Use a real CSV parser rather than splitting every line at a comma. Quoted text can legitimately contain the delimiter, and a quoted field can span physical lines.

Test representative records

Inspect a small sample containing punctuation, quotes, empty fields and non-English text. Confirm that each parsed record has the expected number of fields. Treat a malformed record as something to investigate; skipping bad lines without a report can change the analysis population.

For Python file input, the documented approach uses newline='' with the CSV module. Specify the encoding and dialect appropriate to the actual export. Automatic dialect detection can be a starting hypothesis, but verify its result against the field schema and known examples.

Reproduce a hypothetical failure

Consider the record A17,"pain, mild",3. It contains three fields: identifier A17, the text pain followed by a comma and mild, and the value 3. A naive comma split produces four pieces and can push the final value into a nonexistent column.

Now imagine 100 exported observations, three of which contain embedded newlines inside quoted notes. A physical line counter could report 104 lines including the header, while a correct parser still finds 100 data records. Neither deleting those extra lines nor subtracting one from the line count is a reliable repair.

Reconcile the imported table

Check parsed record count, column count, header names and key uniqueness against the export manifest. Inspect how empty strings and literal missing-value labels are interpreted. A legitimate category such as NA may need different handling from an actual absent value.

Save an import report listing the chosen delimiter, quoting convention, encoding, rejected records and any explicit repairs. Keep repaired output separate from the raw file. Before analysis, round-trip a synthetic record containing a delimiter, quote and newline through the chosen export and import settings. Successful parsing is a structural check; it does not establish that the clinical values are plausible or that the right population was exported.

Source verification: 6 November 2026. Examples are hypothetical.