CSV DictReader loops when used with MySQLdb

Art Zemon art at hens-teeth.net
Tue Mar 9 22:28:54 CET 2010


I'm going crazy.

Using python 2.5.2 on 64 bit Linux.

I have a class which reads CSV files using the CSV DictReader. If I
print the rows, everything works perfectly. If I insert the rows into a
MySQL table using MySQLdb, the DictReader loops back and begins
re-reading from the beginning of the file when it reaches EOF.

Here's the code:

class CsvLoader:
    """load data from a CSV file into a corresponding MySQL table"""

    def __init__(self, fname, schema, db, useCleanReader = False):
        self.schema = schema
        self.db = db
        if useCleanReader:
            self.reader = csv.DictReader(CleanReader(fname),
delimiter=',', quotechar='"')
        else:
            self.reader = csv.DictReader(open(fname), delimiter=',',
quotechar='"')
   
    def loadMysql(self):
        for row in self.reader:
            self.db.insertGeneric(self.schema.tableName(), row)

    def printRows(self):
        for row in self.reader:
            print "\n", row

and here is the insertGeneric method:

    def insertGeneric(self, table, record):
        """execute a generic INSERT, given a dict as input"""
        fieldList = []
        valueList = []
        for k in record.keys():
            fieldList.append(k)
            valueList.append(record[k])
        fields = ", ".join(fieldList)
        m = ['%s']
        valueMarkers = ", ".join(m * len(valueList))  # create a string
like: %s, %s, %s...
        sql = 'insert into %s (%s) values (%s)' % (table, fields,
valueMarkers)
        cursor = self.conn.cursor()
        print "+++ insert: %s <= %s" % (sql, valueList)
        cursor.execute(sql, valueList)
        cursor.close()

useCleanReader is False
CsvLoader.printRows() works fine.
CsvLoader.loadMySql() keeps looping through the CSV file.

Ideas?

    -- Art Z.

-- 

Art Zemon, President
Hen's Teeth Network <http://www.hens-teeth.net/>
Phone: (866)HENS-NET or (636)447-3030 ext. 200
Twitter: AZemon <http://twitter.com/AZemon>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100309/32a34c48/attachment.html>


More information about the Python-list mailing list