[Csv] Please check this out...

[Skip Montanaro]

Gang,

I just checked in an update to csv.py and test/test_csv.py which allows
csv.reader objects to return dicts.  In much the same way that the writer
can write a dict if told what the field name order is, the reader, if given
a list of fieldnames to use as keys can map the incoming list to a
dictionary.

There's just one little hitch.  I see a negative ref count abort in a CVS
debug build *if* there is a typo in the call to csv.reader().  This
short_test.py script demonstrates it on my Mac:

    import sys
    import unittest
    from StringIO import StringIO
    import csv

    class TestDictFields(unittest.TestCase):
        def test_read_short_with_rest(self):
            reader = csv.reader(StringIO("1,2,abc,4,5,6\r\n"), dialect="excel",
                                fieldnames=["f1", "f2"], restfields="_rest")
            self.assertEqual(reader.next(), {"f1": '1', "f2": '2',
                                             "_rest": ["abc", "4", "5", "6"]})

    def _testclasses():
        mod = sys.modules[__name__]
        return [getattr(mod, name) for name in dir(mod) if name.startswith('Test')]

    def suite():
        suite = unittest.TestSuite()
        for testclass in _testclasses():
            suite.addTest(unittest.makeSuite(testclass))
        return suite

    if __name__ == '__main__':
        unittest.main(defaultTest='suite')

Compare the csv.reader() call with the declaration of the __init__ method.
You'll see I've misspelled "restfield", giving it a needless 's'.  This
pushes it into the **options dict, and since that's not an understood
keyword arg, _csv.parser() complains, like so:

    Traceback (most recent call last):
      File "short_test.py", line 9, in test_read_short_with_rest
        fieldnames=["f1", "f2"], restfields="_rest")
      File "/Users/skip/local/lib/python2.2/site-packages/csv.py", line 102, in __init__
        _OCcsv.__init__(self, dialect, **options)
      File "/Users/skip/local/lib/python2.2/site-packages/csv.py", line 93, in __init__
        self.parser = _csv.parser(**parser_options)
    TypeError: 'restfields' is an invalid keyword argument for this function

Under 2.2, all I get is the above traceback (haven't yet tried a 2.2 debug
build).  With the latest CVS and a debug build I get:


    % /usr/local/bin/python short_test.py
    E
    ======================================================================
    ERROR: test_read_short_with_rest (__main__.TestDictFields)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "short_test.py", line 9, in test_read_short_with_rest
        fieldnames=["f1", "f2"], restfields="_rest")
      File "/usr/local/lib/python2.3/site-packages/csv.py", line 102, in __init__
        _OCcsv.__init__(self, dialect, **options)
      File "/usr/local/lib/python2.3/site-packages/csv.py", line 93, in __init__
        self.parser = _csv.parser(**parser_options)
    TypeError: 'restfields' is an invalid keyword argument for this function

    ----------------------------------------------------------------------
    Ran 1 test in 0.029s

    FAILED (errors=1)
    Fatal Python error: Objects/dictobject.c:686 object at 0x476e98 has negative ref count -606348326
    Abort trap

"-606348326" expressed as hex is '0xdbdbdbda' which looks suspiciously like
the 0xdb bytes which debug Pythons scribble in freed memory.

It's time for a long winter's nap here.  I'm sure you'll have it figured out
by the time I check my mail in the morning.  Actually, I'm suspicious
there's a refcounting bug in 2.3a1...

Thx,

Skip