[Python-ideas] CSV api with conversion

[Bruce Leban]

I didn't mean to suggest that errors should be ignored. Making it easy to
map an empty field to None instead of throwing an error isn't ignoring an
error, it's allowing the developer to preserve data. Granted the csv module
doesn't distinguish between empty fields and non-empty fields but many csv
files do have that distinction. Perhaps the csv module should be fixed to
allow that distinction to be available, maybe by adding new Dialects? Aside
from that it's worth considering a variety of use cases before adding
something like this to the standard libraries.

Imagine that I want a missing int in the first field to raise an exception,
in the second to get the value 0 and in the third to get the value None, I
could write something like:
(1) csv.reader(f, (int, (int, None), (int, 0))
(2) csv.reader(f, (int, csv.map_empty(int, None), csv.map_empty(int, 0))

I'm not saying I like either of those syntaxes, I'm just giving examples to
show feasibility. The benefit of either of those is that they can be used
for both reading and writing while an arbitrary function can't.

If I was going to add this conversion feature to csv, it seems to me to make
the most sense to implement this as a Dialect. The documentation for Dialect
is a little thin, but here's an example:

d = Dialect(base_dialect, *ordered_formats, **key_formats)

d = Dialect(csv.excel, int, str, float)
d = Dialect(csv.excel, sku=int, name=str, price=float)

r = csv.Reader(f, dialect=d)

--- Bruce
http://www.vroospeak.com


On Wed, Apr 14, 2010 at 6:45 PM, MRAB <python at mrabarnett.plus.com> wrote:

> Bruce Leban wrote:
>
>> Sure I could do that. I just this think if you're making a convenience
>> function like this you should strive to make it more convenient.  :-)
>>
>>  If you're passing in 'int' as a conversion function then it should do
> what 'int' does, which is to raise an exception.
>
> I could quote the Zen of Python:
>
>    Errors should never pass silently.
>    Unless explicitly silenced.
>
> It shouldn't be so convenient that it leads to sloppy coding. :-)
>
>  --- Bruce
>> (via android)
>>
>>
>>  On Apr 14, 2010 4:03 PM, "MRAB" <python at mrabarnett.plus.com <mailto:
>>> python at mrabarnett.plus.com>> wrote:
>>>
>>> Bruce Leban wrote:
>>> >
>>> > If you do this, you'll probably want to support mapping empty values.
>>> That is...
>>>
>>> The actual values read from the CSV file are strings and you're passing
>>> them to functions:
>>>
>>>   int("1"), str("")
>>>   int(""), str("2")
>>>
>>> If you want it to return a default value instead of raising an exception
>>> on an empty field then you should pass a conversion function which does
>>> that, for example:
>>>
>>>   def int_or_none(field):
>>>       if field.strip():
>>>          return int(field)
>>>       else:
>>>          return None
>>>
>>>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100414/3eaea716/attachment.html>