Am 25.01.2012, 14:31 Uhr, schrieb Stefan Behnel <stefan_ml@behnel.de>:
Dirk Rothe, 25.01.2012 14:16:
I don't know how Firebird handles this, but you should try if
a) it has a direct way to import XML data in some way
b) you can get away with generating "INSERT OR UPDATE" statements.
In general, a large amount of database roundtrips will make your program much slower than a direct import of a database dump, sometimes by orders of magnitude. Generating a SQL dump file and letting the DB load that directly is bound to be much faster.
Assuming the usage of the kinterbasdb firebrid driver, the most efficient way would be to send a list of tuples as parameters for such a Update/Insert statement. The statement would have to be only prepared (pseudo-compiled, whatever) _once_ at the server side. And the number of network roundtrips will also be minimal.
Right, that makes sense for bulk updates.
However, I was referring to the intention to do a diff and incremental updates between the DB content and the XML data, in which case there'd likely be a lot of requests sending data back and forth between the database and the program while parsing the XML file and comparing the corresponding DB entries. You can't really do any bulk access here.
In my experience that leads normally to some form of caching and "incremental" bulky updates. But I guess thats not really lxml related anymore ;)