[New-bugs-announce] [issue39092] Csv sniffer doesn't attempt to determine and set escape character.

Evan report at bugs.python.org
Wed Dec 18 15:50:42 EST 2019

New submission from Evan <evan.whitfield at gmail.com>:

I observed a false positive for the csv sniffer has_header method. (It thought there was a header when there was not.) This is due to the fact that in has_header, it determines the csv dialect by sniffing it, and failed to determine that the file I was using had an escape character of '\'. Since it doesn't set the escape character, it then incorrectly broke the first line of the file into columns, since it encountered an escaped quote within a quoted column, and treated that as the end of that column. (It correctly determined that the dialect wasn't doublequote, but apparently still needs to have the escape character set to handle an escaped quotechar.) 

I think one (or both) of these things should be done here to avoid this false positive:
1.) Allow a dialect to be passed to has_header, so that someone could specify the escape character of the dialect if it were known.
2.) Allow the sniff method of the Sniffer class to detect and set the escapechar.

components: Library (Lib)
messages: 358645
nosy: evan.whitfield
priority: normal
severity: normal
status: open
title: Csv sniffer doesn't attempt to determine and set escape character.
type: behavior
versions: Python 3.9

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list