Mailman integration with Google Groups: allow Groups Subscribers post directly
Hi,
I find it convenient to have a Google Groups interface for Mailman lists. This adds search, thread monitoring at no cost, and makes communication for people who don't like email a lot easier (look at PySide list as an example). There is one problem though:
- if a mirrored list requires subscription, you need to subscribe both to mailing list and to groups -
This takes a lot of fun out of using Groups, but the worst thing that your message may be delivered and will be available in Groups and not in Mailman. So, my question is:
- how can I setup Mailman, so it can accept mails sent through Google Groups proxy automatically? -
Please, CC. Thanks.
anatoly t.
anatoly techtonik wrote:
- how can I setup Mailman, so it can accept mails sent through Google Groups proxy automatically? -
The envelope sender of a google groups post is group_name+some_token@googlegroups.com. If that address were a member of the mailman list with no password reminders and no mail delivery, the posts would be accepted as member posts. The problem is the +some_token suffix to the local part of the address. The Mailman.Message.get_senders() method doesn't treat +some_token as a suffix and since it is variable, it is not possible to subscribe all the possiblegroup_name+some_token@googlegroups.com addresses to the list.
Thus you are left with a couple of other options. If the Google group is set to include a Reply-To: group_name@googlegroups.com header in delivered posts, and if group_name@googlegroups.com is a member of the mailman list (with no password reminders and no mail delivery), Mailman.Message.get_senders() will return that address in the senders list and Mailman will consider the post from a member.
Even if the Google group is not set to include a Reply-To:, it still includes an X-BeenThere: group_name@googlegroups.com header. If you add
SENDER_HEADERS = ('from', None, 'reply-to', 'sender', 'x-beenthere')
to mm_cfg.py, Mailman.Message.get_senders() will consider the address in X-BeenThere: as well as the addresses in From:, the unix from, Reply-To: and Sender: as senders which will be checked against the list membership.
So, if you add group_name@googlegroups.com as a member of the mailman list (with no password reminders and no mail delivery), and either set the google group "Replies to messages" to "Replies are sent to the whole group" or add the above SENDER_HEADERS line to mm_cfg.py, posts arriving via the group will be seen as member posts.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Mar 29, 2011 at 9:49 AM, Mark Sapiro mark@msapiro.net wrote:
anatoly techtonik wrote:
- how can I setup Mailman, so it can accept mails sent through Google Groups proxy automatically? -
The envelope sender of a google groups post is group_name+some_token@googlegroups.com. If that address were a member of the mailman list with no password reminders and no mail delivery, the posts would be accepted as member posts. The problem is the +some_token suffix to the local part of the address. The Mailman.Message.get_senders() method doesn't treat +some_token as a suffix and since it is variable, it is not possible to subscribe all the possiblegroup_name+some_token@googlegroups.com addresses to the list.
Am I right that "envelope sender" is just a relay proxy address, so To: field will still contain my email? Is it possible to add an option to Mailman itself to strip +token before processing email addresses? Does email standard allow to use +token addresses for multiple users?
Thus you are left with a couple of other options. If the Google group is set to include a Reply-To: group_name@googlegroups.com header in delivered posts, and if group_name@googlegroups.com is a member of the mailman list (with no password reminders and no mail delivery), Mailman.Message.get_senders() will return that address in the senders list and Mailman will consider the post from a member.
Seems easy, but isn't it possible to forge Reply-To: address to spam Mailman lists? I'd like to ensure there is some kind protection to avoid spam that pretends to be from the Group. The same concern is about X-BeenThere header solution.
So, if you add group_name@googlegroups.com as a member of the mailman list (with no password reminders and no mail delivery), and either set the google group "Replies to messages" to "Replies are sent to the whole group" or add the above SENDER_HEADERS line to mm_cfg.py, posts arriving via the group will be seen as member posts.
If I set "no mail delivery" then how the Group receive messages for reading from the web?
Please, CC. Thanks.
anatoly techtonik wrote:
On Tue, Mar 29, 2011 at 9:49 AM, Mark Sapiro mark@msapiro.net wrote:
anatoly techtonik wrote:
- how can I setup Mailman, so it can accept mails sent through Google Groups proxy automatically? -
The envelope sender of a google groups post is group_name+some_token@googlegroups.com. If that address were a member of the mailman list with no password reminders and no mail delivery, the posts would be accepted as member posts. The problem is the +some_token suffix to the local part of the address. The Mailman.Message.get_senders() method doesn't treat +some_token as a suffix and since it is variable, it is not possible to subscribe all the possiblegroup_name+some_token@googlegroups.com addresses to the list.
Am I right that "envelope sender" is just a relay proxy address, so To: field will still contain my email?
The envelope sender is set by Google Groups to an address to whih undeliverable type notices will be returned. If I understand correctly what you're doing, the To: header will contain the address of the Google Group. The recipient address (the Mailman list which is a member of the Google Group) will not appear in headers other than perhaps Received:.
Is it possible to add an option to Mailman itself to strip +token before processing email addresses?
Only by modifying code.
Does email standard allow to use +token addresses for multiple users?
I do not understand the question. There will be one message from the Google Group to the Mailman list. That message's envelope sender will have a unique token so if it bounces, Google can easily tell which recipient bounced. The messages from the Google Group to other Google Group members will each have their own unique token.
Thus you are left with a couple of other options. If the Google group is set to include a Reply-To: group_name@googlegroups.com header in delivered posts, and if group_name@googlegroups.com is a member of the mailman list (with no password reminders and no mail delivery), Mailman.Message.get_senders() will return that address in the senders list and Mailman will consider the post from a member.
Seems easy, but isn't it possible to forge Reply-To: address to spam Mailman lists?
Yes, and one can almost as easily forge From: for the same purpose.
I'd like to ensure there is some kind protection to avoid spam that pretends to be from the Group. The same concern is about X-BeenThere header solution.
It doesn't take much sophistication to craft and send an email message with anything one wants in the headers. People who want to can already easily spoof the address of any list member in the From: header of a message to fool Mailman into accepting the post as from a list member.
If you are truly concerned about this, you have to moderate all members to hold all posts and approve them manually.
So, if you add group_name@googlegroups.com as a member of the mailman list (with no password reminders and no mail delivery), and either set the google group "Replies to messages" =C2=A0to "Replies are sent to the whole group" or add the above SENDER_HEADERS line to mm_cfg.py, posts arriving via the group will be seen as member posts.
If I set "no mail delivery" then how the Group receive messages for reading from the web?
No mail delivery is a per user option. You set no mail delivery and no password reminders for only the Google Groups address which is a member of the Mailman list so that Mailman doesn't send the post back to the Google Group. Delivery to other members of the Mailman list is unaffected by this.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat, Apr 16, 2011 at 10:01 PM, Mark Sapiro mark@msapiro.net wrote:
anatoly techtonik wrote:
On Tue, Mar 29, 2011 at 9:49 AM, Mark Sapiro mark@msapiro.net wrote:
anatoly techtonik wrote:
- how can I setup Mailman, so it can accept mails sent through Google Groups proxy automatically? -
The envelope sender of a google groups post is group_name+some_token@googlegroups.com. If that address were a member of the mailman list with no password reminders and no mail delivery, the posts would be accepted as member posts. The problem is the +some_token suffix to the local part of the address. The Mailman.Message.get_senders() method doesn't treat +some_token as a suffix and since it is variable, it is not possible to subscribe all the possiblegroup_name+some_token@googlegroups.com addresses to the list.
Let me concentrate on this part once more. I need only one group to be a frontend for one list. So I need only mails from this group to be delivered - not from all groups. If I understood correctly, +some_token is regenerated every post and it is "envelope address". My own @gmail.com address it not seen by Mailman when the message arrives, so the From: format is "My Name group_name+some_token@googlegroups.com". Is this right?
I am not an administrator and can't validate this information. Just to make clear about the situation. The problem with just subscribing group address that when I post through the group (not being subscribed to the list), my post seems to appear in group only, but not on mailing list. I don't receive any notifications or bounces. But when I subscribe to the list, my post seem to pass through.
Is it possible to add an option to Mailman itself to strip +token before processing email addresses?
Only by modifying code.
Does it qualify as a feature request?
Does email standard allow to use +token addresses for multiple users?
I do not understand the question. There will be one message from the Google Group to the Mailman list. That message's envelope sender will have a unique token so if it bounces, Google can easily tell which recipient bounced. The messages from the Google Group to other Google Group members will each have their own unique token.
I see. I thought that if +token is something optional and that email standard specifies that "email+something@domain.com" is equivalent to "email@domain.com" and denotes the same recipient, and that email agents are not required to process the +part, then Mailman could just ignore it.
Thus you are left with a couple of other options. If the Google group is set to include a Reply-To: group_name@googlegroups.com header in delivered posts, and if group_name@googlegroups.com is a member of the mailman list (with no password reminders and no mail delivery), Mailman.Message.get_senders() will return that address in the senders list and Mailman will consider the post from a member.
The group is set into "Replies are sent to the whole group." mode. But the subscribed address is group_name+token@googlegroups.com. Now I see where the problem is.
I'd like to ensure there is some kind protection to avoid spam that pretends to be from the Group. The same concern is about X-BeenThere header solution.
It doesn't take much sophistication to craft and send an email message with anything one wants in the headers. People who want to can already easily spoof the address of any list member in the From: header of a message to fool Mailman into accepting the post as from a list member.
I thought there is a reverse MX check to validate the message was sent from the IP that belongs to MX record for this domain.
So, if you add group_name@googlegroups.com as a member of the mailman list (with no password reminders and no mail delivery), and either set the google group "Replies to messages" =C2=A0to "Replies are sent to the whole group" or add the above SENDER_HEADERS line to mm_cfg.py, posts arriving via the group will be seen as member posts.
If I set "no mail delivery" then how the Group receive messages for reading from the web?
No mail delivery is a per user option. You set no mail delivery and no password reminders for only the Google Groups address which is a member of the Mailman list so that Mailman doesn't send the post back to the Google Group. Delivery to other members of the Mailman list is unaffected by this.
Still can't understand how Google Group receive messages if Mailman doesn't send anything to its address. Do you mean that "no mail delivery" means that Mailman still sends all messages to Groups address except messages that were posted from this group (containing Group address in Reply-To: header)?
-- anatoly t.
anatoly techtonik wrote:
Let me concentrate on this part once more. I need only one group to be a frontend for one list. So I need only mails from this group to be delivered - not from all groups. If I understood correctly, +some_token is regenerated every post and it is "envelope address". My own @gmail.com address it not seen by Mailman when the message arrives, so the From: format is "My Name group_name+some_token@googlegroups.com". Is this right?
No. You could answer these questions definitively for yourself by just looking at the full headers of any message received from the Google group.
The message from the Google group will have headers
To: <the address of the google group> From: <the name and address of the poster to the google group>
and will be sent with the envelope from (reflected in a Return-Path: header) group_name+some_token@googlegroups.com
I am not an administrator and can't validate this information. Just to make clear about the situation. The problem with just subscribing group address that when I post through the group (not being subscribed to the list), my post seems to appear in group only, but not on mailing list. I don't receive any notifications or bounces. But when I subscribe to the list, my post seem to pass through.
I think you are saying that if you subscribe the Mailman list to the Google group and post to the Google group, your post is sent from the Google group to the Mailman list and is accepted by the Mailman list. If this is the case, and if the Mailman list accepts posts only from list members, then this works because you are a member of the Mailman list. It will also work for any other poster to the Google group who is also a member of the Mailman list. The issue I am trying to help solve is for posts to the Google group from people who are not members of the Mailman list.
Is it possible to add an option to Mailman itself to strip +token before processing email addresses?
Only by modifying code.
Does it qualify as a feature request?
Anything can be requested.
Feature requests can be submitted to the bug tracker at https://bugs.launchpad.net/mailman.
Does email standard allow to use +token addresses for multiple users?
I do not understand the question. There will be one message from the Google Group to the Mailman list. That message's envelope sender will have a unique token so if it bounces, Google can easily tell which recipient bounced. The messages from the Google Group to other Google Group members will each have their own unique token.
I see. I thought that if +token is something optional and that email standard specifies that "email+something@domain.com" is equivalent to "email@domain.com" and denotes the same recipient, and that email agents are not required to process the +part, then Mailman could just ignore it.
The Mail RFCs 2822 and 5322 specify regarding the local-part of an address (the part left of the @)
The local-part portion is a domain dependent string. In addresses, it is simply interpreted on the particular host as a name of a particular mailbox.
I.e., it is up to the receiving domain to assign meaning to any xxx+yyy local part. It is fairly common for the '+yyy' part to be ignored and the 'xxx' part to be treated as the recipient, but as far as I know, there is no standard governing this. Some MTAs use a '-' instead of or in addition to a '+' as the delimiter, some may use other characters and some do not recognize this kind of 'subaddressing' at all.
Thus you are left with a couple of other options. If the Google group is set to include a Reply-To: group_name@googlegroups.com header in delivered posts, and if group_name@googlegroups.com is a member of the mailman list (with no password reminders and no mail delivery), Mailman.Message.get_senders() will return that address in the senders list and Mailman will consider the post from a member.
The group is set into "Replies are sent to the whole group." mode. But the subscribed address is group_name+token@googlegroups.com. Now I see where the problem is.
Good.
I'd like to ensure there is some kind protection to avoid spam that pretends to be from the Group. The same concern is about X-BeenThere header solution.
It doesn't take much sophistication to craft and send an email message with anything one wants in the headers. People who want to can already easily spoof the address of any list member in the From: header of a message to fool Mailman into accepting the post as from a list member.
I thought there is a reverse MX check to validate the message was sent from the IP that belongs to MX record for this domain.
Your own incoming MTA can implement any such tests it wants, and there is a standard (SPF) addressing this, but such tests tend to reject lots of legitimate mail, e.g. mail redirected via a .forward. In any case, such tests when used will test the domain of the envelope sender, not the domain of the From: header.
So, if you add group_name@googlegroups.com as a member of the mailman list (with no password reminders and no mail delivery), and either set the google group "Replies to messages" =C2=A0to "Replies are sent to the whole group" or add the above SENDER_HEADERS line to mm_cfg.py, posts arriving via the group will be seen as member posts.
If I set "no mail delivery" then how the Group receive messages for reading from the web?
No mail delivery is a per user option. You set no mail delivery and no password reminders for only the Google Groups address which is a member of the Mailman list so that Mailman doesn't send the post back to the Google Group. Delivery to other members of the Mailman list is unaffected by this.
Still can't understand how Google Group receive messages if Mailman doesn't send anything to its address. Do you mean that "no mail delivery" means that Mailman still sends all messages to Groups address except messages that were posted from this group (containing Group address in Reply-To: header)?
user@example.com sends a post to group@googlegroups.com. Post is delivered to group members including mailman_list@example.net. Message from group contains Reply-To: group@googlegroups.com and group@googlegroups.com is a member of mailman_list@example.net so message will be accepted by the Mailman list. Do you now want the Mailman list to send the message back to the Google group?
If you wish to follow up further, please say exactly what you want to happen when
- member of Google group who is not member of Mailman list posts to Google group.
- member of Google group who is member of Mailman list posts to Google group.
- member of Mailman list who is not member of Google group posts to Mailman list.
- member of Mailman list who is member of Google group posts to Mailman list.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
anatoly techtonik -
Mark Sapiro