[spambayes-dev] modified version of sb_dbexpimp.py

[Kenny Pitt]

Tony Meyer wrote:
> [Skip]
>> Since this is rather late in the game 1.0-wise I would
>> like a little extra feedback before checking this stuff in.
> 
> I was too late trying it out for this, but it (cvs version) also
> works for me.
> 
> One query (I've used the csv module quite a bit since I moved to 2.3,
> but only reading, never writing, so haven't noticed this before):  I
> see that it writes rows with '\r\n' termination, so in Excel I get
> blank lines between every row (with a file as long as the spambayes
> database, this means I miss a lot of data).
> 
> Should we provide an option to the dbexpimp script to change the line
> terminator to '\n'?  (Simple enough to do, if I read the csv doc
> correctly). Or maybe just have a "if sys.platform == "win32":
> lineterminator = '\n'" kinda thing?

The '\r\n' terminator should be correct for Windows, and if I import my
csv file into Excel 2003 I don't get any blank rows.  Unix, on the other
hand, would normally expect '\n' only.  The usual method for handling
this would be to always specify '\n' as the line terminator, but write
files using text mode instead of binary mode so that the line terminator
is translated according to the current platform.  The csv module doesn't
seem to do it this way, and Skip is opening the output file using 'wb'
mode in sb_dbexpimp which is correct given the way csv is specifying the
line endings.

I did notice one oddity in my exported csv file.  There were a few
tokens, most of them subject: something but a few others, that were
wrapped between two lines using an e-mail header style wrapping where
the continuation line begins with a tab.  Even though the rest of the
file is written with '\r\n' line terminators, these wrapped lines have
only a '\n' terminator.  This anomaly fooled my Vim editor into thinking
the file was a Unix file with '\n' line endings, so I wonder if the same
thing could throw off Excel in some cases.

-- 
Kenny Pitt