Import Decimal Commas Without Changing Numeric Meaning

Establish the export convention

A number written as 1,234 can mean one thousand two hundred thirty-four or one point two three four, depending on the file’s convention. Do not infer its meaning from your computer’s locale. Ask for the export specification or inspect unambiguous examples from the source system before converting text to numbers.

The pandas read_csv reference documents separate delimiter, decimal and thousands settings. These address different parts of the format. A semicolon-delimited file can use a comma decimal separator without ambiguity, while an unquoted comma-delimited field containing a decimal comma may be structurally broken.

Build a discriminating import sample

Choose synthetic values containing a fraction, a negative number, a thousands group and a missing field. Include a value near a planned scientific threshold. Keep the original text column alongside the parsed numeric column until the import rules are accepted.

Check units as well as punctuation. A correctly parsed value in milligrams is still wrong if the analysis assumes grams. Record whether spaces used as grouping marks are ordinary spaces or another Unicode character, and reject unsupported mixtures instead of stripping every nonnumeric character.

Work through a hypothetical threshold

Suppose an export uses decimal commas and semicolons. A row contains P01;1,25;0,75. Under the intended convention the two measurements are 1.25 and 0.75, and their sum is 2.00. Removing commas as if they were grouping marks produces 125 and 75, whose sum is 200.

If a quality rule flags measurements above 10, the wrong import creates two false alerts. More dangerously, a downstream scale conversion could make those values look plausible again. That is why threshold checks should supplement a verified import example, not replace one.

Reconcile the whole column

Count successful conversions, explicit missing values and failures. Review all failures and a sample of successful conversions, particularly extreme values and threshold boundaries. Compare summary ranges with source-system summaries where available, using the same population and units.

Save the import settings in code or a configuration file, not only in a spreadsheet dialog. Include an assertion that the synthetic examples parse as expected. If the export convention changes between sites or releases, import each declared format separately and record the transition. A single permissive parser that guesses row by row can conceal a mixed-format data problem that deserves source correction.

Source verification: 6 November 2026. Examples are hypothetical.