On Sep 20, 2013, at 12:46 PM, Aurelien Bompard wrote:
>What do you think? Bad timing? Any existing work I'm missing?
I think it's a *great* idea.
No, perfect timing.
Not that I know of! :)
What we've said in the past is that we weren't going to promise conversion
scripts from 2.1 to 3.0, partly because that work was incomplete, but also
because we wanted to caution users not to just blindly upgrade as soon as 3.0
is released. But I think having a reliable, well-tested, non-destructive
option to upgrade would be wonderful, especially because I fully hope that a
site could run 2.1 and 3.0 together for a while.
Any help you need, let me know.
(I've been ridiculously swamped at work these past few weeks. Apologies for
being rather quiet lately. I hope that'll improve in the next couple of
FYI, I will be releasing 2.1.16rc1 in a few days with a target of a
final release in early September.
I believe the release will be very solid and stable. The main purpose of
the candidate release is to expose any i18n changes so that translators
can submit any updates and get them in the final release. There are a
few new features, contributed programs, i18n changes and bug fixes, all
of which will be announced when the candidate is released.
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
This is the final progress report for my project, Authenticated REST API
The code is available at https://bitbucket.org/mgill25/mm-rest
I'll go over the major points of the work done so far:
* The public-facing REST Interface is mostly done. Most business use
cases, things like creating/deleting
lists, subscriptions etc are working via REST.
* The REST Interface currently works with Basic Auth, but other
authentication schemes are easily pluggable
via the Django REST framework.
* Postorius has been modified in order to use the MM-REST models instead
of directly interacting with
* Hence, in the case of no connection available to Core, MM-REST, and
continue to work on their own, saving data in their own level. Which
brings me to...
* The Caching Interface b/w Mailman Core and MM-REST, which is yet to be
I have a (somewhat) generic interface, but haven't managed to make it
work for all the models, primarily Memberships.
* Workaround in mm-rest or modifications in Core for allowing backing up
of certain Models,
which is currently limited by the Core REST API. For example:
Registering new Email addresses for a User.
* And there's a small todo file in the project containing some things to
be done yet.
The project is not 100% finished yet, and after the GSoC is formally
over, I shall continue working on it.
Murray S. Kucherawy writes:
> I wonder how long we can hold out before we start trying to drag
> [the MUA developers] into our conversations, which might be the
> only way to solve these pain points long term. It seems to me that
> Gmail, Yahoo Mail, Thunderbird, etc., must have either a team or an
> individual that spends time thinking about and testing user-visible
> solutions to these problems, so perhaps it's time to start asking
> them for help.
To be honest, I don't think they really care. Even larsi, who never
saw an internet-draft he wouldn't implement, is not terribly
systematic about it -- as long as things are smooth between Gnus
users, well, tough luck for the rest of the world. When I was talking
to MUA developers about best practices for dealing with reply-to-list
the basic response was "we already have a function for that" or
"sounds great, patches welcome". The Mozilla people were clearly much
more interested in features like calendars and vcards.
I think it's important we get started on it (and maybe I can
contribute something now that GSoC is almost over), but I don't have
much hope that the MUA developers will put a high priority on it.
It's not really in their purview (see Franck Martin's comments about
MUA irrelevance in this thread, for example, and he's not even an MUA
And of course the worst offender is Microsoft, which AFAICS is
somewhat actively undermining the RFC 822 standard for reasons I don't
On Sep 17, 2013, at 6:21 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 09/17/2013 05:28 PM, Barry Warsaw wrote:
>> On Sep 15, 2013, at 08:24 AM, Mark Sapiro wrote:
>>> Because the issue remains controversial, I will soon release 2.1.16
>>> final with the feature disabled by default, and will consider the
>>> message encapsulation approach or other possibilities based on
>>> experience with 2.1.16 for a 2.1.17 release perhaps early next year.
>> This makes sense to me, although I would label the feature "provisional" or
>> "experimental". There just isn't any good experience here we can draw on, so
>> it seems reasonable to provide the knobs that will allow motivated folks to
>> gather such experience, but generally keep it out of the way for the majority
>> of users.
> I intend to so indicate in the NEWS about the feature.
> I have given some thought to the encapsulation approach and have some
> questions about it.
> My thoughts on how to do it include the following if the feature is enabled.
> CookHeaders saves the original Subject: before prefixing in the metadata.
> A new handler before ToOutgoing but after ToDigest, ToArchive and
> ToUsenet creates a new message From: the list with Content-Type:
> message/rfc822, a unique Message-ID: and Subject:, References: and
> In-Reply-To: copied from the current message. It then replaces the
> Subject: in the current message with that saved in the metadata and sets
> the payload of the new message to the current message.
> This will result in a single-part message with a payload of the content
> filtered original message. If content filtering hasn't removed anything,
> the original message's DKIM signatures if any should still be valid.
> The message then goes to ToOutgoing and eventually OutgoingRunner and
> SMTPDirect which will call Decorate and if any msg_header or msg_footer
> is added, these will be added as is currently done which will result in
> the message becoming multipart/mixed with the addition of one or two
> text/plain, Content-Disposition: inline parts containing the header
> and/or footer.
> The idea is the message/rfc822 part preserves as much of the original as
> possible so its DKIM sigs if any may still validate and to put enough
> into the headers of the new message so MUAs can still thread it properly
> and render it nicely. Also, the message is unchanged over current
> behavior for digests, archives and usenet.
> The sticky questions are what to do with the original From: and
> Reply-To: in the face of possible Reply-To: munging options and so on.
> Presumably, we want to still be able to reply to the original author,
> and preserve the behavior of a simple MUA 'reply' going to the original
> author and not the list in the absence of Reply-To: munging, and that
> could be accomplished by putting the original author's Reply-To: (or the
> original From: if no original Reply-To:) in the new message's Reply-To:,
> but it's not yet clear to me how to handle the munging options (maybe
> just ignore them ;).
> I could actually implement this approach for the 2.1.16 release, but
> that would negate the i18n work that's already been done as the
> descriptive string on General Options would change, so I'm more inclined
> to label this feature as experimental and subject to change
> significantly in a future release.
1) If you keep the From: header as it is then, we will still have the same problems
2) the purpose of the encapsulation message/rfc822 is not meant to preserve the DKIM signature, DKIM is not meant to be verified by MUAs
3) encapsulation is not there to provide a transitive trust, there is another method explored for that which is Original-Authentication-Results (OAR). There is an expired Internet Draft on it.
I think, it is risky to code this encapsulation method directly and now, it requires a branch some testing and then merging back into the main branch.
The author_is_list has had deployment and testing for over a year in a DMARC environment. Limited testing I agree but nevertheless proved not to break things ( besides people ideal view of email ;) ) and be useable with many MTAs and especially MUAs
Here is a recent test, deployment and analysis: http://sys4.de/en/blog/2013/08/11/mailman-dmarc-konform-betreiben/
I'd like to see somebody operating a mailing list with this encapsulation method first, before merging.
I have successfully added Exim 4 support to Mailman 3. The branch is
There's an instance at maxking-test(a)turnbull.sk.tsukuba.ac.jp which
also contains Abhilash's signed-list feature. Since it's very strict
about checking signatures, it probably won't be much fun to use. :-P
Postorius is running at
It's a separate process group from my main site, so the URLs you'll
see in the address bar aren't very pretty, it needs the extra
"postorius" in front to distinguish the top page, and I haven't
bothered to fix the rest of the name space.
No HyperKitty yet, sorry.
So I have pushed my final commit for now for my GSoC project. I will try
to be brief on all the accomplishments and mention concisely
* Signature rule following strict rfc 3156 can now be verified, only
pgp-mime messages are accepted and messages without a (valid) signature
* SignMessage handler creates multipart/signed message, first of whose
part is the original message formed after all the processing(in
default-posting-pipeline this handler is at last before the message is
send out or copied to nntp or any other queue). It expects a
`multipart/signed` or `multipart/mixeed` message. The signature is
calculated on whole part using the list's secret key.
* A List's key can be created from postorius and other signature
parameters can also be changed from there( only 'signature_max_age for
* A user can import his public key from public keyservers( defaults to
ha.pool.sks-keyservers.net) by entering their public key id or they can
copy paste their public key data to upload their public key.( Exceptions
handling is not implemented yet in postorius, if works fine in perfect
I hope to improve all these things even more but seems like I have a
short of time now, I will update you( and the mm-dev list) with what
other plans I have to furthur improve these parts.
P.S.: There is a working version of my copy of mailman here if you
want to have a look.
I'm in the process of testing the conversion from Fedora's current mailman
2.1 instance to a test 3.0 instance. I realize the import21 command is
currently pretty naive and does not migrate most of the settings, nor the
I've looked at bug 965532  and more recently the email thread about
creating such a script during GSOC , and unless I'm mistaken nothing
moved in that general area during the summer. Am I right or did I miss
I'd like to work on that, if that's the right time and no-one else is
working on it.
I'll probably extend the import21 command to convert the other settings. I
understand the tricky part is to convert email addresses in 2.1 to users in
3.0, so that's probably going to be a separate module with its own tests.
At least it'll serve as a point of reference and we'll be able to refine
the use cases and the algorithm later on.
What do you think? Bad timing? Any existing work I'm missing?
In the upcoming mailman 2.1.16 there has been the introduction of the optional feature author_is_list
"Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority. If setting this to Yes, it is advised to set the MTA to DKIM sign all emails. "
Usage of this feature on lists have indicated no adverse issue for the members, provided proper communication is done when the feature is enabled (as to allow inbox rules to be changed if needed).
In the 2.1.16 Release Candidate the feature author_is_list is controlled by the following switches in the mm_cfg.py
ALLOW_AUTHOR_IS_LIST = No
DEFAULT_AUTHOR_IS_LIST = No
The first one will enable the GUI to present an option to the list admin to enable the author_is_list feature, the second controls if new lists or upgraded lists should have the author_is_list feature set to yes
While it is better for the MTA to DKIM sign emails if author_is_list is enabled, this is not a requirement and the list will work fine without DKIM.
While DEFAULT_AUTHOR_IS_LIST is important to avoid new lists and upgraded lists change behavior, setting ALLOW_AUTHOR_IS_LIST to Yes does not change how any list is handled (author_is_list in the GUI is No by default). it only provides an option to the list admin to change the list behavior.
Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL type install), therefore it would be nice to remove ALLOW_AUTHOR_IS_LIST or set it to Yes by default to let the list admin decide how he/she wants the list to behave. Otherwise the list admin needs to contact the mailman admin to enable this config.
Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes or remove this option from mm_cfg.py
Note: a few organizations have indicated they will provide the necessary translation of this feature in the GUI for the most common languages.
When a list goes bad, usually the members are not blamed but the list admin, therefore making the list the system responsible of the writing of the message.
Anyhow, it does not matter, this is a religious discussion. Please feel free to code and test your solution of encapsulating the message in a mime rfc822. This seems an interesting and good alternative. I'd like to see it in practice so we can compare data.
On Sep 14, 2013, at 1:27 AM, Stephen J. Turnbull <stephen(a)xemacs.org> wrote:
> Franck Martin writes:
>> One may argue that since the list is modifying the message, it is
>> now the new author of it, this proposal just make it more clearly.
> Nonsense. Here's what RFC 5322 says:
> The "From:" field specifies the author(s) of the message, that is,
> the mailbox(es) of the person(s) or system(s) responsible for the
> writing of the message.
> The list obviously isn't responsible for the writing of the message
> body, and you could argue that in adding header/footer and munging
> attachments and Subject field it's acting as the agent of the author,
> who is therefore responsible for them too.
> If that's not convincing, ask any of your users if they think "the
> list" is an author of their posts, or anybody else's.
> OTOH, if you want to make an authorship claim validly, there's an easy
> way to accomplish it: encapsulate the whole thing in message/rfc822.
>  Note that RFC 5322's phrasing also clearly refutes the same
> argument when made for "Reply-To".