hi devs!
from calafou hacklab & friends we wanted to reimplement schleuder in python. speaking and speaking we arrived at the conclusion that maybe writing a plug-in for Mailman 3 could be a good solution: we contribute to a great project and we have encrypt mailing list for hacktivist people.
first and obvious question: is there already an ongoing effort to achieve gpg encryption that we could join?
if not:
we thought this plug-in as a milter (mail filter) with added features for executing commands.
after reading your Core docs, things are little bit more complicated:
- the preprocess (and maybe postprocess?) of the messages could be done
by what you call
chainsandpipelines - the command system could be implemented extending the already
existing Mailman command system (
echoandend). - it is just not clear how to prompt if encripton is desired in your ecosystem.
our questions are maybe:
- is there a plug-in way to tackle this problem?
- do we really need to submit a merge request to the core instead of doing an optional debian package?
- are we totally on the wrong way? :)
thx for the support! best wishes!
efkin. Calafou.
Hi,
from calafou hacklab & friends we wanted to reimplement schleuder in python. speaking and speaking we arrived at the conclusion that maybe writing a plug-in for Mailman 3 could be a good solution: we contribute to a great project and we have encrypt mailing list for hacktivist people.
imho a good idea. afaik is Schleuder undergoing a "v2 rewrite" - I have the feeling they discovered it's not that easy to write a mailinglist manager (even without encryption) ;-)
first and obvious question: is there already an ongoing effort to achieve gpg encryption that we could join?
I tried to do that ~10 years ago. I tried to implement an addition to the pipeline chain. The patch was further maintained by Joost van Baal-Ilic, who reverted the pipeline approach and made it a direct patch for Mailman. As far as I can tell, the project is resting (I haven't checked for a long time, though).
imho the pipeline chain approach is the preferable approach. I remember I still had to do some minor patches at the Mailman core because some operations weren't doable otherwise (plus the configuration settings). Unfortunately, I haven't had time to have a closer look at Mailman 3, so I can't tell if things have become easier there.
- do we really need to submit a merge request to the core instead of doing an optional debian package?
My personal recommendation would be: Stick to separate modules (i.e. the pipeline approach) as long as possible, this should make things easier to maintain:
- the whole encryption stuff is a lot more tricky than one might think at first sight
- you don't want to add instability for the majority (unfortunately) of users who don't want to use any encryption on their mailing list
Good luck, I'd love to see someone picking up that usecase!
Stefan.
On 11/11/2015 09:28 AM, Stefan Schlott wrote:
Hi,
from calafou hacklab & friends we wanted to reimplement schleuder in python. speaking and speaking we arrived at the conclusion that maybe writing a plug-in for Mailman 3 could be a good solution: we contribute to a great project and we have encrypt mailing list for hacktivist people.
imho a good idea. afaik is Schleuder undergoing a "v2 rewrite" - I have the feeling they discovered it's not that easy to write a mailinglist manager (even without encryption) ;-)
first and obvious question: is there already an ongoing effort to achieve gpg encryption that we could join?
I tried to do that ~10 years ago. I tried to implement an addition to the pipeline chain. The patch was further maintained by Joost van Baal-Ilic, who reverted the pipeline approach and made it a direct patch for Mailman. As far as I can tell, the project is resting (I haven't checked for a long time, though).
wow 10 years ago! can u give me a link to the project? so as far i understood by the archive, recurrently in time somebody appears saying that he/she 's gonna develop this feature. but there was this funny answer that said "this feature may be implmented in mailman 3". so maybe we are at the right moment in the right place :)
imho the pipeline chain approach is the preferable approach. I remember I still had to do some minor patches at the Mailman core because some operations weren't doable otherwise (plus the configuration settings). Unfortunately, I haven't had time to have a closer look at Mailman 3, so I can't tell if things have become easier there.
- do we really need to submit a merge request to the core instead of doing an optional debian package?
My personal recommendation would be: Stick to separate modules (i.e. the pipeline approach) as long as possible, this should make things easier to maintain:
- the whole encryption stuff is a lot more tricky than one might think at first sight
- you don't want to add instability for the majority (unfortunately) of users who don't want to use any encryption on their mailing list
so, as far as i understood, your proposal is actually to write separate modules that import actually existing modules and add functionality over them?
it seems reasonable to me...
Good luck, I'd love to see someone picking up that usecase!
thx.
Stefan.
efkin.
Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/efkin%40cooperati...
Security Policy: http://wiki.list.org/x/QIA9
On Nov 11, 2015, at 09:28 AM, Stefan Schlott wrote:
I tried to do that ~10 years ago. I tried to implement an addition to the pipeline chain. The patch was further maintained by Joost van Baal-Ilic, who reverted the pipeline approach and made it a direct patch for Mailman. As far as I can tell, the project is resting (I haven't checked for a long time, though).
imho the pipeline chain approach is the preferable approach. I remember I still had to do some minor patches at the Mailman core because some operations weren't doable otherwise (plus the configuration settings). Unfortunately, I haven't had time to have a closer look at Mailman 3, so I can't tell if things have become easier there.
An important difference between MM2 and MM3 is that in MM2, both moderation and modification functions were intertwined in a single pipeline-of-handlers mechanism. This turned out to be to unwieldy so in MM3, we split moderation into a pre-processing chain of links-and-rules, and modification into a post-acceptance pipeline of handlers. The latter looks a lot like MM2's pipeline of handlers, except of course that it does not make moderation decisions.
Cheers, -Barry
efkin writes:
from calafou hacklab & friends we wanted to reimplement schleuder in python.
URLs would be nice.
first and obvious question: is there already an ongoing effort to achieve gpg encryption that we could join?
Not really, although there's been discussion of it; see below. The first big problem is use case; there are several and they demand somewhat different treatment. I suppose your use case is defined by "schleuder", but I don't know what that is, and the first followup suggests that they aren't 100% sure themselves. :-)
after reading your Core docs, things are little bit more complicated:
- the preprocess (and maybe postprocess?) of the messages could be done by what you call
chainsandpipelines
Just pipelines, most likely. The basic idea is that chains determine whether messages are accepted for delivery at all, pipelines handle transformations and actual delivery to various targets.
- the command system could be implemented extending the already existing Mailman command system (
echoandend).
What command system? Do you mean the REST interface, or the GSoC project to provide a nicer CLI?
- it is just not clear how to prompt if encripton is desired in your ecosystem.
It is, at least by users. There's been a GSoC project to add some signature and encryption capabilities (IIRC only signatures were implemented), but it's not been merged yet.
http://www.google-melange.com/gsoc/project/details/google/gsoc2013/maxking/5...
Abhilash, do you have anything to say? ;-)
- is there a plug-in way to tackle this problem?
What do you mean by "plug-in"?
- do we really need to submit a merge request to the core instead of doing an optional debian package?
Given that you're intervening in the pipeline in a big way (have you considered what your "plugin" might imply for DKIM and DMARC, as well as various existing transformations that Mailman can apply?) and you say you need to "extend the command system", I think the latter would have to be considered a fork. I doubt you'd get any support from the Debian maintainership or Mailman core if you do that.
- are we totally on the wrong way? :)
Hard to tell. I supervised the GSoC project mentioned above, but I have no idea what you're talking about with respect to "schleuder". That means there's a good chance nobody in core knows much, either.
So tell us about it, and then we'll tell you. :-)
On 11/11/2015 11:51 AM, Stephen J. Turnbull wrote:
efkin writes:
from calafou hacklab & friends we wanted to reimplement schleuder in python.
URLs would be nice.
After some minimalist tackle to the problem we decided to start a small research about the problem here[0]. And schleuder software is here[1].
[0] https://gitlab.com/calafou/lythyr/issues/1 [1] http://schleuder2.nadir.org/
first and obvious question: is there already an ongoing effort to achieve gpg encryption that we could join?
Not really, although there's been discussion of it; see below. The first big problem is use case; there are several and they demand somewhat different treatment. I suppose your use case is defined by "schleuder", but I don't know what that is, and the first followup suggests that they aren't 100% sure themselves. :-)
after reading your Core docs, things are little bit more complicated:
- the preprocess (and maybe postprocess?) of the messages could be done by what you call
chainsandpipelinesJust pipelines, most likely. The basic idea is that chains determine whether messages are accepted for delivery at all, pipelines handle transformations and actual delivery to various targets.
Interesting.
- the command system could be implemented extending the already existing Mailman command system (
echoandend).What command system? Do you mean the REST interface, or the GSoC project to provide a nicer CLI?
something like what is described here: http://mailman.readthedocs.org/en/release-3.0/src/mailman/commands/docs/end.... it could be helpful, for example, to the mantainers of the encrypted list to add one fingerprint/email to the mailing list object without having to have shell access to the machine where it is installed. maybe i just misunderstood what these commands are. but the idea is that a mantainer could execute specific commands given a special syntax in the firstline of an email.
- it is just not clear how to prompt if encripton is desired in your ecosystem.
It is, at least by users. There's been a GSoC project to add some signature and encryption capabilities (IIRC only signatures were implemented), but it's not been merged yet.
http://www.google-melange.com/gsoc/project/details/google/gsoc2013/maxking/5...
Abhilash, do you have anything to say? ;-)
- is there a plug-in way to tackle this problem?
What do you mean by "plug-in"?
well, basically something that could be packaged as mailman3-lythyr at does not need to patch the core. i think Stefan suggested a modular approach. and it seems quite reasonable.
- do we really need to submit a merge request to the core instead of doing an optional debian package?
Given that you're intervening in the pipeline in a big way (have you considered what your "plugin" might imply for DKIM and DMARC, as well as various existing transformations that Mailman can apply?) and you say you need to "extend the command system", I think the latter would have to be considered a fork. I doubt you'd get any support from the Debian maintainership or Mailman core if you do that.
we obviously want to avoid a fork. at the same time, the three of us, that would work on this project, are quite unaware of mailman3 and mailman in general architecture. but at the same time we are enthusiast of researching it and happy to code for it.
- are we totally on the wrong way? :)
Hard to tell. I supervised the GSoC project mentioned above, but I have no idea what you're talking about with respect to "schleuder". That means there's a good chance nobody in core knows much, either.
well, hope the links i gave at the beginning are enough. if you need some more info just let us know.
So tell us about it, and then we'll tell you. :-)
On Nov 11, 2015, at 12:48 PM, efkin wrote:
What do you mean by "plug-in"?
well, basically something that could be packaged as mailman3-lythyr at does not need to patch the core. i think Stefan suggested a modular approach. and it seems quite reasonable.
The intent of Mailman 3 is that additions like this could be packaged as separate modules, and through config file change could be pulled in and used by the core. The reality is that the core may need some code changes so that all the necessary plug points could be exposed. This is part of what I said before about multiple merge proposals.
Cheers, -Barry
Just to add to what others have mentioned. "Encrypted mailing lists" mean different things to different people, so being precise about the features you're interested in will be key. There are very likely some fundamental work that would need to be done in any case (e.g. connecting pubkeys to users).
On Nov 11, 2015, at 12:29 AM, efkin wrote:
after reading your Core docs, things are little bit more complicated:
- the preprocess (and maybe postprocess?) of the messages could be done by what you call
chainsandpipelines
Chains are used in the moderation process, i.e. making the decision "should this message get posted to the mailing list". So if you wanted to make posting decisions based on message signatures, there would be a rule to parse the signature, verify it, match it to a user, and register a rule "hit" if it matches. The link containing this rule might have a jump action to immediately accept (i.e. jump to the accept chain) it if the signature matched. Rules are not allowed to modify the message.
If "encrypted lists" are a special thing, there might even be a named chain that can be used instead of the default chain for those mailing lists that need this feature.
Pipelines come into play *after* the message has been accepted for posting. They contain handlers which modify the message in some way. So if "encrypted mailing list" means to sign the message with a list key, this could happen in a handler. Note that handlers can currently only handle modifications that would affect all recipients of the message. For personalized modifications (e.g. VERPing or re-encrypting the message to individual recipient's pubkey), some changes to the code would have to happen; ideally we'd be able to define pipelines that would be called at MTA delivery time.
- do we really need to submit a merge request to the core instead of doing an optional debian package?
This feature, once properly scoped, isn't going to be a "one merge request" submission. There are multiple fundamental changes that could happen along the way, and some of those might be appropriate for core.
Cheers, -Barry
participants (4)
-
Barry Warsaw -
efkin -
Stefan Schlott -
Stephen J. Turnbull