[Mailman-Users] Mailman + giant lists + the infinite weight of the cosmos
brad at stop.mail-abuse.org
Tue Feb 8 21:31:10 CET 2005
At 11:52 AM -0800 2005-02-08, Carl Zwanzig wrote:
>> 3. Any general advice for handling ridiculously large lists?
> You might want to look at http://www.python.org/doc/faq/library.html#id35,
> otherwise, I think you're hacking MM to default to the marshal method. The
> code is there, but as the backup method. (Or really hacking it to use
Using Berkeley db is pretty blindingly fast, and if done right,
should result in an extremely reliable database. MySQL uses
BerkeleyDB as their way of achieving full ACID compliance in their
high-end MaxSQL database product. Rock-damn-bloody-solid stuff, if
done right. But also easier to screw up.
If you can replace the marshal method with one based on Berkeley
DB, I imagine that will probably solve most of the performance
problems right there. Assuming you choose the right type of file
format, that is. I imagine that a btree is likely to perform better
in this application than a hash table.
Alternatively, a Berkeley DB-based member adaptor should not be
too excessively difficult to code. However, if such a beast doesn't
already exist, you might want to first try the LDAP member adaptor.
Brad Knowles, <brad at stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755
SAGE member since 1995. See <http://www.sage.org/> for more info.
More information about the Mailman-Users