[Mailman-Developers] about qrunner and locking

[Chuq Von Rospach]

>     TW> professional years of experience in this area, NFS sucks large
>     TW> granite elephant testicles through a very thin straw.

Now that's a visual I did not need... (grin) {completely off the 
subject, I was part of the group that worked on the first third party 
portsof NFS to non-sun hardwar,e back in the days when I was working 
at sun... Bonus points for knowing what the first non-sun hardware 
NFS ran on...)

>Background for those who don't know: zodb is the Zope Object Database,
>ZEO is Zope Enterprise Objects.

My only worry about this is adding enough complexity and overhead 
that mailman loses it's attractiveness to the small site.

>Pythonlabs.  We've talked about all this stuff before, but the
>question now is: is it better to jump in sooner rather than later?

Probably sooner, if that's the direction we want to go -- but that 
simply defines 2.1 as "bug fixes and really easy stuff", and puts us 
in 3.0 development sooner, rather than later So to a good degree it 
means 2.1 or 3.0 determinations are made based on "easy" rather than 
"high priority" to minimize re-doing stuff when it's rearchitected.

>We'd have to handle collisions for multiple qrunner processes,
>potentially on separate machines.  One way that doesn't involve
>locking shenanigans is to divide the hash space up and assign a
>segment to each out-qrunner process.

here's another way that should work: each record has a locking field 
in it. When qrunner wants to execute that item, it reads the field. 
If the field is NULL, it writes its ID (hwatever it is, guaranteed 
unique) into that locking field. It then waits a beat, and reads it 
back. if it reads back its own ID, it knows it owns the record and 
can execute it. If it reads back someone else's ID, it lost the lock, 
but someone else owns the record so it can skip it and move on.

you can simulate atomic locks with a little thought and cooperative 
processes, by everyone writing to the store and then seeing who won. 
A LOT easier from and administrative view than partitioning hashes 
and the like, IMHO.

-- 
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com)
Apple Mail List Gnome (mailto:chuq@apple.com)

We're visiting the relatives. Cover us.