[Tutor] finally without try or except

[Don Jennings]

On Jul 31, 2012, at 12:26 PM, tutor-request at python.org wrote:

> Message: 2
> Date: Tue, 31 Jul 2012 10:44:29 -0400
> From: Tino Dai <oberoc at gmail.com>
> To: "Steven D'Aprano" <steve at pearwood.info>
> Cc: "tutor at python.org" <tutor at python.org>
> Subject: Re: [Tutor] finally without try or except
> Message-ID:
> 	<CAOu0yXadVv9gjbQjf1LKRcXPiwAu4j7K4RF_sGrCQukP4fgE9A at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> On Mon, Jul 30, 2012 at 9:01 PM, Steven D'Aprano <steve at pearwood.info>wrote:
> 
>> If you want to be robust, it is best not to try to beat the database. That
>> means you should write to the database as soon as you can, as often as you
>> need to, and let the database do what it does best: reliable transaction
>> storage. Any decent database will guarantee ACID compliance (atomicity,
>> consistency, isolation, durability).
>> 
> 
>> But sometimes you need to compromise on robustness for speed or
>> convenience.
>> 
>> 
>> [...]
>> 
>> This is how the program was originally structured, but we found
>>> performance
>>> problems with in.
>>> It's using Django and an Oracle DB, which is notoriously bad for single
>>> record read/writes. So, I
>>> figured why don't we append them all to an array, and write them out at
>>> the
>>> end of the program.
>>> 
>> 
>> I suggest you periodically check the item log, and if there are more than
>> (say) 20 items, you write them to the database and clear the temporary
>> list. That way, the number of item logs should never get too large, and the
>> performance shouldn't suffer too greatly. You will need to tweak that
>> number to be more or less depending on how poorly the Oracle DB performs.
>> 
>> Then, at the very end of the program, you write whatever items are left in
>> a finally clause, or atexit. That way, the finally clause should be nice
>> and speedy and the user is unlikely to hit Ctrl-C a second time.
>> 
>> 
> I think that is the process I'm going to take. It doesn't seem as elegant,
> but I think it's a good compromise.


Another option might be to write each item to a different database which has better performance for single record writes—and, I'm betting someone on this list will have an excellent recommendation for such a DB. Then, you could batch the updates to the Oracle DB independently of the main program (the one related to your question).

Take care,
Don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/c0ef01d2/attachment.html>