Re: [Mailman-Developers] Common use case archiving via configuration
Thanks Aurélien
Would you mind if I used the Hyperkitty POST code as the basis for an example POST archiver for Mailman?
i.e. this: https://github.com/hyperkitty/mailman-hyperkitty/blob/master/mailman_hyperki...
I’m happy to rewrite from scratch but it seems to make sense to use what you have.
as
On 24 Mar 2015, at 12:45 pm, Aurelien Bompard <aurelien@bompard.org> wrote:
Aurelien I’d be interested to hear your thoughts on this topic.
Well, the implementation of the archiver for HyperKitty does POST to archive emails : https://github.com/hyperkitty/mailman-hyperkitty/blob/master/mailman_hyperki... However, there are a few things specific to HyperKitty: the front-facing server name.
- authentication is done via a shared API key, it was easier than HTTP basic auth because those tokens are not passed to the WSGI application by default (at least not in Apache's mod_wsgi): https://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIPassAutho...
- when requesting a message URL, the server only replies with the "local" part, the server name is taken from the configuration. The reason is that the client usually talks over localhost, and we want
Doing something more generic is possible but would be a bit harder, and the current archiving plugin is still rather short. I'm not sure we can design an HTTP API for every need at the moment.
Aurélien
@barry - would you mind confirming please you’re OK with this please?
I’m going to implement some very simplistic, working, example archivers. As discussed with Stephen, although functional, these will be “working examples”, much like prototype.py is currently, and won’t be presented as complete archive integration solutions, rather as starting points/examples that happen to work for some common use cases. This is to avoid a concern that they might create a maintenance burden if end users see them as an official solution to archive integration.
—> archiver filename: archiving/to_maildir.py -> config options: config[‘archiver.name’].folderpath = —> description: this is close to a straight copy of prototype.py and archives messages to a Maildir
—> archiver filename: archiving/to_filesystem.py -> config options: config[‘archiver.name’].folderpath = config[‘archiver.name’].zip = True -> description: this writes emails to the file system, optionally zipped
—> archiver filename: archiving/to_httppost.py -> config options: config[‘archiver.name’].target_url = config[‘archiver.name’].headers = config[‘archiver.name’].url_parameters = config[‘archiver.name’].form_fields = config[‘archiver.name’].X_Message_ID_Hash_only = -> description: this (with Aurelien’s permission) will be based on Hyperkitty’s HTTP POST archiver and will behave in a similar manner. Alternatively It’ll be written from scratch.
—> archiver filename: archiving/to_smtp.py -> config options: config[‘archiver.name’].destination_address = -> description: this is very close to a straight copy of mail_archive.py but will remove a few lines of code to make it a more generalised smtp sender
—> archiver filename: archiving/to_nntp.py -> config options: config[‘archiver.name’].server_address = config[‘archiver.name’].username = config[‘archiver.name’].password = -> description: this will use the standard library to post the archive message to an NNTP server
—> archiver filename: archiving/to_git.py -> config options: config[‘archiver.name’].foldername -> description: this will write the email to a git repository on the local file system
—> archiver filename: archiving/to_AMQP.py -> config options: config[‘archiver.name’].server_address config[‘archiver.name’].username = config[‘archiver.name’].password = config[‘archiver.name’].X_Message_ID_Hash_only = config[‘archiver.name’].max_message_size = -> description: this will write the message to an AMQP server and optionally writes only the message id hash
On Mar 24, 2015, at 05:01 PM, Andrew Stuart wrote:
@barry - would you mind confirming please you’re OK with this please?
These are all pretty interesting. We'd have to think about whether we'd want some, any, or all them in the main tree, which essentially means we're committing to supporting them.
Alternatively, we can create a contrib directory or repo somewhere where semi-official additions like this could be managed. The idea being that folks could grab these, add a little configuration magic or drop them in the right file system location, and they'd get the new functionality. Doing it this way would also validate or flesh out our plugin story.
Cheers, -Barry
Barry Warsaw writes:
These [example? archivers] are all pretty interesting. We'd have to think about whether we'd want some, any, or all them in the main tree, which essentially means we're committing to supporting them.
Alternatively, we can create a contrib directory or repo
Batteries included, please!
A nice option would be that in the admin interface, the admin could tick a box saying "show me contrib archivers" and then tick and configure the appropriate one. The instructions for contrib configurations would say that Mailman is not committed to supporting these but we'll help as time permits (and we always have time to review patches -- a lie, but with good intent <wink />).
On Mar 25, 2015, at 01:07 PM, Stephen J. Turnbull wrote:
A nice option would be that in the admin interface, the admin could tick a box saying "show me contrib archivers" and then tick and configure the appropriate one. The instructions for contrib configurations would say that Mailman is not committed to supporting these but we'll help as time permits (and we always have time to review patches -- a lie, but with good intent <wink />).
An easy way to do that would be to include something like "[provisional]" (thinking PEP 411) or "[unsupported]" in the archiver name. This should then be visible to the list admin when they're selecting them.
Cheers, -Barry
Agreed Stephen.
It would be good if only configuration was required to drive the archivers even though they are only examples/provisional/contrib whatever.
I’m hoping that projects like Hyperkitty and my server won’t need installers to touch the Mailman source code at all to get data flowing from Mailman.
Once you understand archivers they’re pretty easy but to newbies I suspect they are pretty intimidating to grasp.
What does "batteries included" mean in this context?
as
On 25 Mar 2015, at 3:07 pm, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Barry Warsaw writes:
These [example? archivers] are all pretty interesting. We'd have to think about whether we'd want some, any, or all them in the main tree, which essentially means we're committing to supporting them.
Alternatively, we can create a contrib directory or repo
Batteries included, please!
A nice option would be that in the admin interface, the admin could tick a box saying "show me contrib archivers" and then tick and configure the appropriate one. The instructions for contrib configurations would say that Mailman is not committed to supporting these but we'll help as time permits (and we always have time to review patches -- a lie, but with good intent <wink />).
Andrew Stuart writes:
What does "batteries included" mean in this context?
That you shouldn't need to fetch the archivers from a separate archive or repo to configure them. Sure, you *could* register them as plugins and have a button to fetch them from PyPI (or just do it if the user selects one, but I don't see a huge benefit to that practice in this case.
Thus I prefer a contrib directory or subtree to the repo proposal.
participants (4)
-
Andrew Stuart -
Aurelien Bompard -
Barry Warsaw -
Stephen J. Turnbull