Re: [Mailman-Developers] Gsoc Support for Message queue based email archiver
On Sun, Mar 13, 2016 at 16:22:25PM +0530, Anirudh Dahiya wrote:
Great to hear you're interested in this project. I've just responded to another question regarding this project that is equally relevant to you:
http://www.mail-archive.com/mailman-developers%40python.org/msg16305.html
Sounds good. If you're already reading through the HyperKitty source, you might give special attention to its implementation of Mailman's IArchiver interface, as well as the implementations for the mail-archive.com and mhonarc archivers in the Mailman core projects.
Florian
Hello I have been getting to know more about the message queues and the pub/sub systems available. As I am trying to formalize the project description for the proposal, I have a few doubts regarding this.
Reliability - Currently, as I see, the messages are archived in Hyperkitty via a simple POST request, but in the event of some failure(like hk server being down for a while), there is no way the message is being tried to be archived again. Thus are we aiming to add some reliability to the system via the message queue systems? In our use case, I believe reliability is a critical issue as we make threads from individual mails in the archiver. Apart from this and extensibility, what all advanteges are we looking forward to as a result of this project? In Zeromq, I am still searching for a storage based message queue with multiple consumers. Also, the pub/sub model is based around the idea of message being dropped (as in subscribers not listening to the broadcaster miss out on the messages.) I found an interesting approach based in redis https://davidmarquis.wordpress.com/2013/01/03/reliable-delivery-message-queu...
Should I add/mention the potential scope for adding mailman events (like list creation etc) to our archivers?
Am I supposed to alter Hyperkitty(as what seems to be mentioned in the project description) or should the basic scope of the project stay about building the pub/sub or message queue interface(as what seems to be mentioned by some of our conversations on irc)
I do partially know answers to some of the questions above, but would like a confirmation and a concrete idea about the project before I apply my proposal. Thank you for your time Regards Anirudh Dahiya (irc spark)
On Tue, Mar 15, 2016 at 3:00 PM, Florian Fuchs <f@florianfuchs.com> wrote:
Hello I have been getting to know more about the message queues and the pub/sub systems available. As I am trying to formalize the project description for the proposal, I have a few doubts regarding this.
Reliability - Currently, as I see, the messages are archived in Hyperkitty via a simple POST request, but in the event of some failure(like hk server being down for a while), there is no way the message is being tried to be archived again. Thus are we aiming to add some reliability to the system via the message queue systems? In our use case, I believe reliability is a critical issue as we make threads from individual mails in the archiver. Apart from this and extensibility, what all advanteges are we looking forward to as a result of this project? In Zeromq, I am still searching for a storage based message queue with multiple consumers. Also, the pub/sub model is based around the idea of message being dropped (as in subscribers not listening to the broadcaster miss out on the messages.) I found an interesting approach based in redis https://davidmarquis.wordpress.com/2013/01/03/reliable-delivery-message-queu...
Should I add/mention the potential scope for adding mailman events (like list creation etc) to our archivers?
Am I supposed to alter Hyperkitty(as what seems to be mentioned in the project description) or should the basic scope of the project stay about building the pub/sub or message queue interface(as what seems to be mentioned by some of our conversations on irc)
I do partially know answers to some of the questions above, but would like a confirmation and a concrete idea about the project before I apply my proposal. Thank you for your time Regards Anirudh Dahiya (irc spark)
On Tue, Mar 15, 2016 at 3:00 PM, Florian Fuchs <f@florianfuchs.com> wrote:
participants (2)
-
Anirudh Dahiya -
Florian Fuchs