Message queue based archiver - Requesting opinions/suggestions on design architecture

Hi I am working on implementing message queue based archiving system as part of my GSoC project. Just yesterday, me and Florian(my mentor) were discussing possible architectural choices, and decided to approach the mailman-developer community for opinions.
Right now, the archiving happens something like this - [image: Plugin.png]
We were discussing the architectural design for the proposed system to be plugged. We considered two choices -
I. The first one we looked at involved an additional interface between the backends and IArchiver. The job of this interface would be to decide to which backend to approach for a particular archive, and subsequently send the mail to that backend.
The advantage I saw here was that archives/webapps that would be developed in future would then be able to attach to the our system, and not disturb IArchiver.
II. To this, Florian suggested, and I realized later after some thought, that choosing backend + the archive right at the IArchiver stage seems more sensible. This information can be stored in mailman.cfg, following the current way it works for archiving. It seems more simple and follows the way archiving is done up untill now.
I later realised that since we'll have to add info about which archive subscribes to which mailing lists(actually stored the other way round), in our first design, we would eventually have to add information at the "Our interface" level. Thus better to do all this at one place, i.e. the IArchiver interface level, which the second design proposes.
One thing I learnt from reading backends' documentation is that they encourage several clients(archives in our case) plugged into the same message queue system, rather than defining a queue system for each archive. This saves space, and processor time(for the backend server running in the background).
These are our thoughts, I'd love to hear opinions and suggestions to design choices.
Regards Anirudh Dahiya (irc spark)
participants (1)
-
Anirudh Dahiya