Multiple modules with database access + general app design?

[Robin Haswell]

On Thu, 19 Jan 2006 12:23:12 +0000, Paul McGuire wrote:

> "Robin Haswell" <rob at digital-crocus.com> wrote in message
> news:pan.2006.01.19.10.28.37.668978 at digital-crocus.com...
>> Hey people
>>
>> I'm an experience PHP programmer who's been writing python for a couple of
>> weeks now. I'm writing quite a large application which I've decided to
>> break down in to lots of modules (replacement for PHP's include()
>> statement).
>>
>> My problem is, in PHP if you open a database connection it's always in
>> scope for the duration of the script. Even if you use an abstraction layer
>> ($db = DB::connect(...)) you can `global $db` and bring it in to scope,
>> but in Python I'm having trouble keeping the the database in scope. At the
>> moment I'm having to "push" the database into the module, but I'd prefer
>> the module to bring the database connection in ("pull") from its parent.
>>
>> Eg:
>> import modules
>> modules.foo.c = db.cursor()
>> modules.foo.Bar()
>>
>> Can anyone recommend any "cleaner" solutions to all of this?
> 
> Um, I think your Python solution *is* moving in a cleaner direction than
> simple sharing of a global $db variable.  Why make the Bar class have to
> know where to get a db cursor from?  What do you do if your program extends
> to having multiple Bar() objects working with different cursors into the db?
> 
> The unnatural part of this (and hopefully, the part that you feel is
> "unclean") is that you're trading one global for another.  By just setting
> modules.foo.c to the db cursor, you force all Bar() instances to use that
> same cursor.
> 
> Instead, make the database cursor part of Bar's constructor.  Now you can
> externally create multiple db cursors, a Bar for each, and they all merrily
> do their own separate, isolated processing, in blissful ignorance of each
> other's db cursors (vs. colliding on the shared $db variable).

Hm if truth be told, I'm not totally interested in keeping a separate
cursor for every class instance. This application runs in a very simple
threaded socket server - every time a new thread is created, we create a
new db.cursor (m = getattr(modules, module)\n m.c = db.cursor() is the
first part of the thread), and when the thread finishes all its actions
(of which there are many, but all sequential), the thread exits. I don't
see any situations where lots of methods will tread on another method's
cursor. My main focus really is minimising the number of connections.
Using MySQLdb, I'm not sure if every MySQLdb.connect or db.cursor is a
separate connection, but I get the feeling that a lot of cursors = a lot
of connections. I'd much prefer each method call with a thread to reuse
that thread's connection, as creating a connection incurs significant
overhead on the MySQL server and DNS server.

-Rob

> 
> -- Paul