Hi, I'm administering and moderating a list with Mailman version 2.1.20. I need to make the following changes with immediate effect:
- Switch to the Postfix email router from Sendmail,
- Use Office 365 with authentication as the relay host
I have stopped the Sendmail service and configured Postfix for email transactions. Things are not working as they used to while using Sendmail. I have specifically noticed that the following Mailman functions are not working after the change over:
- Online membership registrations are not receiving email communication about the membership being held for approval.
- The moderator does not receive communication about new online registrations. So, new online registrations are not happening.
- Adding new subscriptions through the command line works, but neither the subscriber nor the moderator receives any communication. Ex. ./add_members -r new -w y -a y listname
- No confirmation email is received after posting a message by a subscribed member.
- The <from> email address of the approved messages is changing to the authentication email id, but the name remains that of the member making the post.
Could someone guide me to resolve the above issues, please?
Thanking you,
- Francis
Francis Jayakanth via Mailman-Users writes:
Hi, I'm administering and moderating a list with Mailman version 2.1.20.
This is extremely old. If it works, that's fine, but you're missing 19 releases worth of security fixes, including some quite nasty and easily exploitable ones (like cross-site scripting).
What does "configure for email transactions" mean? We really can't help unless you're quite precise about these things. Where does the O365 relay host sit? Between Mailman and Postfix, or between Postfix and the Internet?
Is O365 involved in transmitting these notifications? If so, is the sending agent (Mailman or Postfix) configured to use the authentication credentials? What do the logs say? Both Mailman and the MTAs should be keeping logs of all outgoing messages.
Does the post go out to the subscribers? Are you sure this notification is enabled? (In my experience members hate it with a passion). Again, this should be logged by Mailman and both MTAs.
Is it possible that this issue is affecting receipt of notifications because recipients have not whitelisted the authenticated address, and they're being discarded or quarantined as spam? Is the authenticated address in the same domain as Mailman, and if not, do you have a DMARC policy of p=reject or p=quarantine?
That sounds like something that O365 is doing. Mailman should not touch the From email address unless you have configured one of the following:
- from is list (normally used to work around DMARC issues)
- anonymous list
- full personalization
Stock Mailman will not use connection creditials to modify the message; it only becomes aware of them at the point it actually connects to the MTA.
Steve
Thank you, Stephen, for the reply and valuable tips. I will upgrade Mailman to the latest version asap. I have an urgent solution for the following issue.
Postfix is configured to use the O365 as a relay host in our Mailman use case in our Mailman use case. One of our lists has close to 6k members. There were two messages for approval. I approved one and received it for both my list subscriptions. After about an hour or so, I approved the second message. The second message was not delivered to any of the subscribers. The reason is that the O365 email id used for relay host authentication was blocked with the following statement in the O365 Admin console:
"Potentially compromised account. Anomalous sending patterns were detected, resulting in the user being restricted sending mails"
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365. How can the said restrictions be complied with in Mailman?
Thanks in advance, Francis
From: Stephen J. Turnbull <stephenjturnbull@gmail.com> Sent: 01 August 2022 19:42 To: Francis Jayakanth <francis@iisc.ac.in> Cc: mailman-users@python.org <mailman-users@python.org> Subject: [Mailman-Users] Change in email routing
External Email
Francis Jayakanth via Mailman-Users writes:
Hi, I'm administering and moderating a list with Mailman version 2.1.20.
This is extremely old. If it works, that's fine, but you're missing 19 releases worth of security fixes, including some quite nasty and easily exploitable ones (like cross-site scripting).
What does "configure for email transactions" mean? We really can't help unless you're quite precise about these things. Where does the O365 relay host sit? Between Mailman and Postfix, or between Postfix and the Internet?
Is O365 involved in transmitting these notifications? If so, is the sending agent (Mailman or Postfix) configured to use the authentication credentials? What do the logs say? Both Mailman and the MTAs should be keeping logs of all outgoing messages.
Does the post go out to the subscribers? Are you sure this notification is enabled? (In my experience members hate it with a passion). Again, this should be logged by Mailman and both MTAs.
Is it possible that this issue is affecting receipt of notifications because recipients have not whitelisted the authenticated address, and they're being discarded or quarantined as spam? Is the authenticated address in the same domain as Mailman, and if not, do you have a DMARC policy of p=reject or p=quarantine?
That sounds like something that O365 is doing. Mailman should not touch the From email address unless you have configured one of the following:
- from is list (normally used to work around DMARC issues)
- anonymous list
- full personalization
Stock Mailman will not use connection creditials to modify the message; it only becomes aware of them at the point it actually connects to the MTA.
Steve
Francis Jayakanth via Mailman-Users writes:
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365.
I'm not sure what "limit of 30 emails/minute" means. In the below, I am going to assume it means "addresses to be delivered." The other meaning I could imagine would be "connections", which would make it much easier to comply (as long as you have a few "giant" destinations like Gmail and Yahoo).
How can the said restrictions be complied with in Mailman?
There is no facility for this in Mailman itself. Mailman does maintain queues, but their purpose is only to ensure that messages are processed by each function in order and do not get lost while waiting for processing. It contains no logic for "fair queuing" or "throttling" for individual outgoing messages. It just sends them all to the MTA (Postfix), with popular domains getting multiple addressees and only one message body. The only restriction implemented in Mailman is the maximum number of addressees per message. That is maybe you have 1500 Gmail addresses, then you could limit to 25 addressees per SMTP transaction, to allow 5 other emails to get through every minute.
Normally I would recommend using Postfix to do the throttling you need (see the various "recipient_limit" and "rate_delay" parameters in postconf(5)), but given this requirement:
One of our lists has close to 6k members.
you are in a bad place no matter how you look at it unless you can throttle the *incoming* posts to 4-6 per day, spaced at least 3 and probably 4 hours apart. Once one post is in the queue, I don't think there is any way to guarantee it will be sent to all addressees before the next post starts to be sent. So unless you can guarantee posts spaced out in time, you could end up in a situation where 1/4 of the list gets the post, then you wait until the hour, but before that another post sneaks in and it gets delivered to the same 1/4 of the list. That is as far as I know an MTA goes through the recipient domains in a deterministic order, and will start over on the domains that have already had post #1 delivered, by delivering post #2 to them. And of course processing just one post for this list is going to make it difficult for anything else to get delivered until it's done.
Of course the whole time this is going on, you have to keep all of the queued posts on disk, one copy plus the address list per domain. For that reason it would be nice if Mailman handled the queuing for you, but it doesn't.
This is a hard limit (30 emails per minute, 1800 per hour per mailbox) built into O365. Microsoft reccomends using 3rd party bulk mail oroviders like MailChimp. O365 mailboxes are not really suitable for mass emailing, and you can only relay through a desgnated mailbox, iirc.
This isn’t fixable with O365 as the relay host afaik. If the list server can be made public facing so that Postfix manages outgoing email outright (a lot more work because its’ another public-facing server, along with any security implications)
We use O365, but our SMTP is handled by Barracuda which provides anti-spam, anti impersonation and other services, and we can exempt specific hosts/addresses from these limits with that.
On Aug 2, 2022, at 7:49 AM, Stephen J. Turnbull <stephenjturnbull@gmail.com<mailto:stephenjturnbull@gmail.com>> wrote:
Francis Jayakanth via Mailman-Users writes:
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365.
I'm not sure what "limit of 30 emails/minute" means. In the below, I am going to assume it means "addresses to be delivered." The other meaning I could imagine would be "connections", which would make it much easier to comply (as long as you have a few "giant" destinations like Gmail and Yahoo).
-- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group
Institutions do not have opinions, merely customs
Francis Jayakanth via Mailman-Users writes:
Hi, I'm administering and moderating a list with Mailman version 2.1.20.
This is extremely old. If it works, that's fine, but you're missing 19 releases worth of security fixes, including some quite nasty and easily exploitable ones (like cross-site scripting).
What does "configure for email transactions" mean? We really can't help unless you're quite precise about these things. Where does the O365 relay host sit? Between Mailman and Postfix, or between Postfix and the Internet?
Is O365 involved in transmitting these notifications? If so, is the sending agent (Mailman or Postfix) configured to use the authentication credentials? What do the logs say? Both Mailman and the MTAs should be keeping logs of all outgoing messages.
Does the post go out to the subscribers? Are you sure this notification is enabled? (In my experience members hate it with a passion). Again, this should be logged by Mailman and both MTAs.
Is it possible that this issue is affecting receipt of notifications because recipients have not whitelisted the authenticated address, and they're being discarded or quarantined as spam? Is the authenticated address in the same domain as Mailman, and if not, do you have a DMARC policy of p=reject or p=quarantine?
That sounds like something that O365 is doing. Mailman should not touch the From email address unless you have configured one of the following:
- from is list (normally used to work around DMARC issues)
- anonymous list
- full personalization
Stock Mailman will not use connection creditials to modify the message; it only becomes aware of them at the point it actually connects to the MTA.
Steve
Thank you, Stephen, for the reply and valuable tips. I will upgrade Mailman to the latest version asap. I have an urgent solution for the following issue.
Postfix is configured to use the O365 as a relay host in our Mailman use case in our Mailman use case. One of our lists has close to 6k members. There were two messages for approval. I approved one and received it for both my list subscriptions. After about an hour or so, I approved the second message. The second message was not delivered to any of the subscribers. The reason is that the O365 email id used for relay host authentication was blocked with the following statement in the O365 Admin console:
"Potentially compromised account. Anomalous sending patterns were detected, resulting in the user being restricted sending mails"
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365. How can the said restrictions be complied with in Mailman?
Thanks in advance, Francis
From: Stephen J. Turnbull <stephenjturnbull@gmail.com> Sent: 01 August 2022 19:42 To: Francis Jayakanth <francis@iisc.ac.in> Cc: mailman-users@python.org <mailman-users@python.org> Subject: [Mailman-Users] Change in email routing
External Email
Francis Jayakanth via Mailman-Users writes:
Hi, I'm administering and moderating a list with Mailman version 2.1.20.
This is extremely old. If it works, that's fine, but you're missing 19 releases worth of security fixes, including some quite nasty and easily exploitable ones (like cross-site scripting).
What does "configure for email transactions" mean? We really can't help unless you're quite precise about these things. Where does the O365 relay host sit? Between Mailman and Postfix, or between Postfix and the Internet?
Is O365 involved in transmitting these notifications? If so, is the sending agent (Mailman or Postfix) configured to use the authentication credentials? What do the logs say? Both Mailman and the MTAs should be keeping logs of all outgoing messages.
Does the post go out to the subscribers? Are you sure this notification is enabled? (In my experience members hate it with a passion). Again, this should be logged by Mailman and both MTAs.
Is it possible that this issue is affecting receipt of notifications because recipients have not whitelisted the authenticated address, and they're being discarded or quarantined as spam? Is the authenticated address in the same domain as Mailman, and if not, do you have a DMARC policy of p=reject or p=quarantine?
That sounds like something that O365 is doing. Mailman should not touch the From email address unless you have configured one of the following:
- from is list (normally used to work around DMARC issues)
- anonymous list
- full personalization
Stock Mailman will not use connection creditials to modify the message; it only becomes aware of them at the point it actually connects to the MTA.
Steve
Francis Jayakanth via Mailman-Users writes:
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365.
I'm not sure what "limit of 30 emails/minute" means. In the below, I am going to assume it means "addresses to be delivered." The other meaning I could imagine would be "connections", which would make it much easier to comply (as long as you have a few "giant" destinations like Gmail and Yahoo).
How can the said restrictions be complied with in Mailman?
There is no facility for this in Mailman itself. Mailman does maintain queues, but their purpose is only to ensure that messages are processed by each function in order and do not get lost while waiting for processing. It contains no logic for "fair queuing" or "throttling" for individual outgoing messages. It just sends them all to the MTA (Postfix), with popular domains getting multiple addressees and only one message body. The only restriction implemented in Mailman is the maximum number of addressees per message. That is maybe you have 1500 Gmail addresses, then you could limit to 25 addressees per SMTP transaction, to allow 5 other emails to get through every minute.
Normally I would recommend using Postfix to do the throttling you need (see the various "recipient_limit" and "rate_delay" parameters in postconf(5)), but given this requirement:
One of our lists has close to 6k members.
you are in a bad place no matter how you look at it unless you can throttle the *incoming* posts to 4-6 per day, spaced at least 3 and probably 4 hours apart. Once one post is in the queue, I don't think there is any way to guarantee it will be sent to all addressees before the next post starts to be sent. So unless you can guarantee posts spaced out in time, you could end up in a situation where 1/4 of the list gets the post, then you wait until the hour, but before that another post sneaks in and it gets delivered to the same 1/4 of the list. That is as far as I know an MTA goes through the recipient domains in a deterministic order, and will start over on the domains that have already had post #1 delivered, by delivering post #2 to them. And of course processing just one post for this list is going to make it difficult for anything else to get delivered until it's done.
Of course the whole time this is going on, you have to keep all of the queued posts on disk, one copy plus the address list per domain. For that reason it would be nice if Mailman handled the queuing for you, but it doesn't.
This is a hard limit (30 emails per minute, 1800 per hour per mailbox) built into O365. Microsoft reccomends using 3rd party bulk mail oroviders like MailChimp. O365 mailboxes are not really suitable for mass emailing, and you can only relay through a desgnated mailbox, iirc.
This isn’t fixable with O365 as the relay host afaik. If the list server can be made public facing so that Postfix manages outgoing email outright (a lot more work because its’ another public-facing server, along with any security implications)
We use O365, but our SMTP is handled by Barracuda which provides anti-spam, anti impersonation and other services, and we can exempt specific hosts/addresses from these limits with that.
On Aug 2, 2022, at 7:49 AM, Stephen J. Turnbull <stephenjturnbull@gmail.com<mailto:stephenjturnbull@gmail.com>> wrote:
Francis Jayakanth via Mailman-Users writes:
I'm told that there are per minute and per hour restrictions of 30 and 1800 emails respectively (inbound and outbound) on o365.
I'm not sure what "limit of 30 emails/minute" means. In the below, I am going to assume it means "addresses to be delivered." The other meaning I could imagine would be "connections", which would make it much easier to comply (as long as you have a few "giant" destinations like Gmail and Yahoo).
-- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group
Institutions do not have opinions, merely customs
participants (3)
-
Bruce Johnson -
Francis Jayakanth -
Stephen J. Turnbull