[New-bugs-announce] [issue5455] csv module no longer works as expected when file opened in binary mode

[Skip Montanaro]

New submission from Skip Montanaro <skip at pobox.com>:

I just discovered that the csv module's reader class in 3.x doesn't work
as expected when used as documented.  The requirement has always been
that the CSV file is opened in binary mode so that embedded newlines in
fields are screwed up.  Alas, in 3.x files opened in binary mode return
their contents as bytes, not unicode strings which are apparently not
allowed by the next() builtin:

% python3.1
Python 3.1a0 (py3k:70084M, Feb 28 2009, 20:46:48) 
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import csv    
>>> next(csv.reader(open("f.csv", "rb")))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_csv.Error: iterator should return strings, not bytes (did you open the 
file in text mode?)
>>> next(csv.reader(open("f.csv", "r")))
['col1', 'col2', 'color']

At the very least the documentation for the csv.reader class is no
longer correct.  However, I can't see how you can open a CSV file in
text mode and not screw up embedded newlines.  I think binary mode
*has* to stay and some other way of dealing with bytes has to be found.

----------
components: Library (Lib)
message_count: 1.0
messages: 83350
nosy: skip.montanaro
nosy_count: 1.0
severity: normal
status: open
title: csv module no longer works as expected when file opened in binary mode
type: behavior
versions: Python 3.0, Python 3.1

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue5455>
_______________________________________