[Mailman-Developers] Incoming Queue format

[Brad Knowles]

At 11:19 PM -0400 9/28/06, emf wrote:

>  I can't find a filesystem that has a filename dependency for inode
>  caching, so I suspect I'm completely misunderstanding this. Could you
>  expand on that a bit?

Some filesystems implement an in-memory hash of recently accessed 
files, but the filenames are typically truncated to fourteen 
characters, and the paths to the files may likewise be truncated.

>  Maybe; but there are at least two filesystems (XFS, reiserfs) and likely
>  more that handle file renaming/creating really cheaply, and have their
>  own ninja ways of dealing with really large directories that are the
>  product of a rather large amount of coding hours.

XFS and ReiserFS do not comprise the entire universe of all 
filesystems in the world in which Mailman will be operated.

There will be plenty of BSD, Solaris, HP-UX, MacOS X, and other OSes 
where Mailman will be used, and even on Linux you're much more likely 
to run into ext2fs or ext3fs than either XFS or ReiserFS on most of 
the several hundred distributions that are available.

>  Maildir has the advantage of being bog standard and readily
>  comprehended. While I'm all in favor of some lmtp delivery mechanism, I
>  don't see why we should continue inventing our own queue-on-disk
>  approach merely to cater to poorly designed filesystems.

While XFS and ReiserFS may have their advantages (and XFS on SGI Irix 
is much better than XFS on Linux), we can't assume that any portion 
of the Mailman community will be using these kinds of filesystems. 
We must be more conservative in our estimates of what filesystem 
features will be available, and code accordingly.

If we were to assume that everyone had XFS, then let's assume they 
all have XFS on Irix, or even Veritas VxFS.

>  It seems to me like anyone likely to end up with a huge enough incoming
>  mailman queue to care about Maildir's inefficiencies would also be able
>  to put a sensible filesystem underneath it.

That may simply not be possible.  Moreover, I have some real 
operational problems with both XFS on Linux and ReiserFS, and I would 
not run a production mail system using them.  Maybe IBM's JFS, if I 
were forced to run a production mail system on Linux at all, but 
certainly not XFS or ReiserFS.

To be honest, I wouldn't run a real production mail system on 
anything less than Veritas VxFS, and I'd be real choosy about my 
underlying hardware, too -- think Hitachi, not EMC.


So your assumptions about what kinds of filesystems may or may not be 
appropriate are not necessarily going to coincide with the decisions 
that other people make, or the kinds of hardware and OS they may be 
forced to live with.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.