How to find bad row with db api executemany()?
python.list at tim.thechases.com
Sat Mar 30 04:17:50 CET 2013
On 2013-03-29 21:19, Roy Smith wrote:
> We're doing it all in one transaction, on purpose. We start with
> an initial dump, then get updates about once a day. We want to
> make sure that the updates either complete without errors, or back
> out cleanly. If we ever had a partial daily update, the result
> would be a mess.
Having had to do some similarly-sized bulk data loads (in my case,
MSSqlServer at $JOB) couple other ideas occur to me:
1) I believe MySQL has a side-loading function (I'd have to go
digging for it; a quick google suggests a "LOAD DATA INFILE"
statement) that allows you to load data from an external file such
as an XML or CSV file
2) Load into a temp table in testable batches, then do some sort of
batch insert into your main table. Again, a quick google suggest the
"INSERT ... SELECT" syntax
More information about the Python-list