Python3 + sqlite3: Where's the bug?
rosuav at gmail.com
Thu Dec 20 16:55:51 CET 2012
On Fri, Dec 21, 2012 at 2:20 AM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> Hmm, but this:
> result = cursor.fetchone()
> yield result
> Works nicely -- only the fetchmany() makes the example break.
Okay, now it's sounding specific to sqlite. I'll bow out. :)
>> Would it spoil your performance improvements to do all the fetchmany
>> calls before yielding anything?
> Well this would effectively then be a fetchall() call -- this is
> problematic since the source data is LARGE (spekaing of gigabytes of
> data here).
That would be a "yes", then. Scratch that!
>> Alternatively, can you separate the
>> two by opening a separate database connection for the foo-reading (so
>> it isn't affected by the commit)?
> At that point in the code I don't actually have a filename anymore,
> merely the connection. But shouldn't the cursor actually be the
> "correct" solution? I.e. in theory, should the example work at all or am
> I thinking wrong?
You say "db.commit()", not "cur2.commit()", so I don't see that a
cursor would un-break what part-way commits is breaking.
> Because if I'm approaching this from the wrong angle, I'll have no
> choice but to change all that code to open separate connections to the
> same file (something that currently are no provisions for).
Is that an sqlite limitation, or just one of your code?
I poked around at the sqlite3 docs, but didn't find any obvious
"clone" option on the connection, nor a way to retrieve the file name.
That would have been fairly convenient. Oh well.
More information about the Python-list