On Tue, 2004-01-27 at 13:40, John Dennis wrote:
Doesn't the new SYNC_AFTER_WRITE flag address this issue? Here is the doc for it:
# This flag causes Mailman to fsync() its data files after writing and # flushing its contents. While this ensures the data is written to disk, # avoiding data loss, it may be a performance killer. Note that this flag # affects both message pickles and MailList config.pck files.
Note that this warning /may/ be superstition. When I did some tests a long while ago, I saw something like a 95% hit in performance on ext3/RH9. Since then, I've been told that others have seen much less of a performance hit, and I've also heard that RH9 is particularly prone to performance problems when under heavy I/O.
It would be nice for folks out there to enable SYNC_AFTER_WRITE on heavy traffic sites and report back on performance. Maybe we should enable this option by default.
Also, I now know how to cut the number of files created and unlinked by Mailman in half. Currently, the qrunners create a .msg and .db file for every message in the queues. I can collapse that to one file, and I think I can do this while still maintaining the Python 2.1 compatibility requirement. I think the upgrade procedure will be fairly straightforward, so I'm seriously considering implementing this for Mailman 2.1.5. It's an important change, but it's mostly internal and I think it would be a big enough win to slip it into a bug fix release. There are other advantages, such as getting rid of those pesky "lost data files for filebase" messages.
-Barry