How to find bad row with db api executemany()?
Tim Chase
python.list at tim.thechases.com
Fri Mar 29 23:17:50 EDT 2013
On 2013-03-29 21:19, Roy Smith wrote:
> We're doing it all in one transaction, on purpose. We start with
> an initial dump, then get updates about once a day. We want to
> make sure that the updates either complete without errors, or back
> out cleanly. If we ever had a partial daily update, the result
> would be a mess.
Having had to do some similarly-sized bulk data loads (in my case,
MSSqlServer at $JOB) couple other ideas occur to me:
1) I believe MySQL has a side-loading function (I'd have to go
digging for it; a quick google suggests a "LOAD DATA INFILE"
statement[1]) that allows you to load data from an external file such
as an XML or CSV file
2) Load into a temp table in testable batches, then do some sort of
batch insert into your main table. Again, a quick google suggest the
"INSERT ... SELECT" syntax[2]
-tkc
[1]
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
[2]
http://dev.mysql.com/doc/refman/5.0/en/insert-select.html
More information about the Python-list
mailing list