[Mailman-Developers] Requirements for a new archiver
Peter C. Norton
spacey-mailman at lenin.nu
Wed Oct 29 23:35:14 EST 2003
On Thu, Oct 30, 2003 at 05:00:48AM +0100, Brad Knowles wrote:
> SGIs XFS on Irix does a pretty good job, with hashed directory
> structures, and an extent-based journaling filesystem. Regretfully,
> I don't think that all of these features are fully supported under
> the Linux version of XFS, and that work has basically ground to a
> halt with the lay-offs of all the key SGI people who had been working
> on XFS. Veritas VxFS also does a good job in this area.
[ A cursory google search indicates that hashed dirs, extents, and
journalling are all in linux xfs. I can't imagine an unsupported
feature making its way into the filesystem that SGI is putting on its
latest and greatest systems, but if you know about this, please share ]
In the case of a one-file-per-message approach, my experience with
vxfs is that it creates a rather slow filesystem when you get your
filesystem to the point of haing with a few hundred thousand small
files (lots of wasted space in the extents and I believe, though I may
be wrong, that there were lots of metadata lookups through multiple
layers of indirections slowing things down).
However reiserfs was built to handle a mix of lots of small files, ala
maildir or mh spools.
I'm not too current on current bsd going-ons, but I'd bet that ffs2
has something to offer in this arena, too, since it looks like it
almost does extent-based allocation now.
> Kirk McKusick and Eric Allman agree with you that this is a
> proper filesystem problem that should be solved at the filesystem
> level (at least, that's what they've said to me when I brought this
> issue up to them), and they feel you should not attempt to solve
> filesystem problems with "tricks" like INN timecaf/timehash cycbufs.
Err... then to relate this to a prior post, why not just use maildirs
on filesystems that are engineered to handle that sort of thing?
> However, while that's nice in theory, that doesn't necessarily
> help us here in the real world.
Unless you are using a filesystem that works for this, right? Like
xfs, vxfs, reiserfs, and probably ffs2. I believe that linux's ext3
has support for hashing directories (or soon will - I don't precisely
know as I've been focusing on other things)
The 5 year plan:
In five years we'll make up another plan.
Or just re-use this one.
More information about the Mailman-Developers