[Mailman-Developers] Grackle archive framework

[Stephen J. Turnbull]

On Sun, Mar 18, 2012 at 2:25 PM, Aamir Khan <syst3m.w0rm at gmail.com> wrote:

> As for searching the archive, there are solutions like Elastic Search,
> Solr, lucene. Can we use one of them to search directly through the maildir.

Not quickly.  Many archives will have thousands of messages, some will have
hundreds of thousands.  There's no reasonable way to avoid indexes when
you've got more than "dozens" of files.  The stats alone will kill performance.

If you want a full-text solution, you're going to have to use a lot of
disk space
(typically as much as the compressed archives themselves, and for some
databases, more).

Also, I don't really see why a few hours' delay for indexing recent messages
would be a problem.