On 9/18/17 1:20 AM, Stephen J. Turnbull wrote:

Terry . writes:

...

Sorry for the delay in responding, and thanks for your generous offer of working with my webhost and/or cPanel to solve this. I passed that offer to my webhost, but it seems they have been able to sort it out with cPanel themselves.

...

I then tested lists in all 7 domains, and they all sent subscribe/unsubscribe emails to me perfectly. I then asked Jim Dory (who has participated in this thread) to setup a default address for his list, and it worked for him, too.

Great!!

Yes. The bottom line here seems to be that things are now working as they should be, so the issue seems to be solved.

...

========================================== The issue with @mydomain.com was caused due to the setting "Discard the email while your server processes it at SMTP time with an error message" under cPanel>>Email>>Default Address. We've setup "Forward to Email Address" to catchall@mydomain.commailto:catchall@mydomain.com. As cPanel support explained, this indicates that all mail that is delivered, but does not have an address (like mailman-bounces@) on this server will be delivered to the default account - this can potentially pose the risk of the email account receiving email for accounts that do not exist, something commonly seen when a domain is being spoofed. Otherwise, it will be rejected with "No such user here".

...

I’m confused by the wording of the above paragraph from the webhost, but maybe they mean that the server is configured to not allow emails to be sent out *from* addresses which can’t receive emails, and this is to help reduce outbound spam.

I'm confused too. Perhaps something got lost or confused in transmission from cPanel via the web host, or perhaps this is some unusual cPanel thing that we don't understand. We may never understand "why" this fix works, but as long as it does work, we can be happy about that.

...

...

So, I guess when Mailman tries to send a subscribe/unsubscribe notification email out from mailman-bounces@mydomain.commailto:mailman-bounces@mydomain.com to the list owner address, maybe the server blocks it, since that mailman-bounces@mydomain.commailto:mailman-bounces@mydomain.com address doesn’t exist, as such.

There is a 'mailman' list and as such, the mailman-bounces address should be deliverable, but cPanel does things in a strange way. Most lists are associated with a domain - perhaps one of several domains hosted on the server. Exim is configured to know this, so that mail to yourlist@yourdomain actually gets delivered to a list named yourlist_yourdomain and mail to otherlist@otherdomain is delivered to a list named otherlist_otherdomain. This is how cPanel handles multiple domains without requiring listnames to be globally unique.

However, the 'mailman' list is different. It's internal name doesn't have a '_domain' suffix so the list 'mailman' exists, but 'mailman_yourdomain' does not.

I suspect cPanel may have had some Exim magic to deal with this that got lost or broken in a cPanel update. That might explain things.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/02/2017 05:37 PM, Terry . wrote:

I don’t understand why either. Maybe the address is supposed to exist in Exim, and not be visible via cPanel's interface (e.g. under "Forwarders"). But it makes me wonder whether everything will work as designed with Mailman under the current version of cPanel (11.66.0.23).

cPanel's Mailman is a kludge. Many things don't work as a non-cPanel Mailman user would expect. This is a result of cPanel's patches that allow lists of the same name in different domains to exist in a single Mailman installation. This in turn is necessary for them to be able to offer Mailman in more or less turnkey, multi-domain hosting environments which in the long run has been good for Mailman by allowing hosting services to offer Mailman to not highly sophisticated customers.

Unfortunately it has also led to a number of situations where Mailman is available to a customer of a cPanel hosting service, whose admins have no interest in supporting Mailman.

The most major change in cPanel is the list name. A list named mylist in the example.com domain is really named mylist_example.com and a list named mylist in the example.net domain is really named mylist_example.net.

This leads to confusion and some minor things like Sibling lists either not working or requiring apparent list addresses like mylist_example.com@example.com.

For the ordinary email case, the Exim Mailman router knows to deliver mail to mylist(-*)@example.com to the mylist_example.com list and mail to mylist(-*)@example.net to the mylist_example.net list.

However, this all breaks down with 'mailman@example.com' because there is only one 'mailman' site list and its name is 'mailman', not 'mailman_example.com'.

Thus, since this used to work, something has changed in cPanel's Exim config so that mail with envelope from mailman-bounces@example.com is no longer deliverable because mailman-bounces@example.com is not a valid address (mailman-bounces@the.canonical.host.domain might be).

However, I'm confused because I know there are a couple of cPanel Mailman hosting services who's admins are on this list and are very conscientious, and they don't seem to see this issue.

If I create a new list (e.g. bugtest3@mydomain.com, as I did today), the only forwarder that cPanel creates (and allows me to see) is: owner-bugtest3@mydomain.com which forwards emails to bugtest3-owner@mydomain.com.

This "forwarder" is a kludge to allow mail to owner-bugtest3@mydomain.com to be delivered to the bugtest3@mydomain.com owner because someone at cPanel thinks that that address should work that way even though Mailman itself only exposes bugtest3-owner@mydomain.com ad an owner address.

As I said, normal mail delivery to list addresses is handled by cPanel's Mailman router, not by aliases or "forwarders"

If any emails actually get sent to mailman-bounces@mydomain.com, then I assume they'll end up in my catchall@mydomain.com mailbox, (whether I have the "Default Address" set to that catchall address, or I have a forwarder forwarding emails from mailman-bounces@mydomain.com to some mailbox), right??? I assume mailman isn't going to look in that catchall mailbox for anything, so do those emails still get processed because of what Mark said about Exim?

I think all the above is correct. As far as Mailman processing those bounces is concerned, they are normally just forwarded to the site list owner. If they go to your catchall address instead, that's probably better.

I received a couple of "Bounce action notification" emails this week, due to list members' mailboxes being over quota, and they seemed to come through to me OK. They were sent from "MyListName [mailman-bounces@serverxyz.mywebhost.com] on behalf of mailman@somedomain.com" to mylistname-owner@mydomain.com.

Is there anything else I can test to ensure all is working OK in regard to administrative addresses which mailman receives emails at?

I'm sure the admin addresses for your lists work just fine. You can test by sending mail to them. -bounces is problematic, but -owner should go to the owner and -request, -confirm, -subscribe (-join), -unsubscribe (-leave) should all send some kind of reply.

It's only 'mailman-bounces' (and other 'mailman' list addresses) that's problematic.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Terry . writes:

Any comments on any of this, Mark or anyone else, especially re this claim: "...this is a result of an upstream design choice from Mailman not from cPanel,

As I understand it, the "design choice" meant is to have a sitewide address "mailman@site.tld". This isn't so much a "design choice" as a long-established Internet mail tradition that there needs to be a contact address that reaches humans for every automatic installation. For the mail system itself, this is formalized in RFC 2142, which defines addresses like "postmaster" and "hostmaster", as well as the "LIST-request" address for mailing lists. Since Mailman has an additional layer of "site" administration above the lists themselves, we added *one address per mailman instance*, the "mailman" address.

Mailman was designed for "real" sites with a single domain hosting lists, not for virtual hosting. This is unfortunate for cPanel, we admit, but handling of the "mailman" list and its associated aliases in Mailman 2 (which is a 15-year-old architecture IIRC) are well- adapted to that use case. (Making it a list is a natural choice, and I don't see how that causes additional difficulties for cPanel.)

It is certainly true that "Mailman does not create an alias for mailman-bounces." Mailman doesn't create *any* aliases, because alias management is done by MTAs. Mailman is agnostic about MTAs, and each MTA has its own system for setting up aliases. Furthermore, many hosts have unique needs for their systems, so there is no "one size fits all" configuration for targets of aliases. We do provide sample configurations for simple cases (single host, single instance, site owner manages most lists too) for the MTAs we are most familiar with, but the responsibility for setting up aliases is with the system administrator who installs Mailman. (This division of responsibilities remains in Mailman 3.)

That "system administrator" might be a person, or it might be a distribution script. I don't know how cPanel is architected, so I don't know where this reponsibility might best be handled in a cPanel installation. But I can tell you that Mailman has never assumed it, while all of the usual distros (Debian, Ubuntu, Red Hat, Centos, FreeBSD, etc) do, each in its own way.

I get the feeling that this doesn't fully address the conversation among you, your host, and cPanel. It should give you some idea of how we view the system administration responsibilities, though.

Hope this helps.

Steve

On 10/23/2017 10:21 PM, Terry . wrote:

cPanel responded with the following 2 emails:

Response #1:

====================================

Hello,

Unfortunately though, as I mentioned previously this is a result of an upstream design choice from Mailman not from cPanel, we had an open inquiry for our development team as I mentioned previously as well which addressed this. This isn't a modification we made in the behavior.

In fact it is a modification that cPanel made to require sender-verify.

In response to point a) A default address is created for every account on the server automatically, I did suggest in my previous response that it is a bad idea to utilize this as a resolution.

In regard to the rest of the points I think it is useful to stress the following:

This isn't an issue that cPanel created, this is an issue that is present due to the utilization of two incompatible configurations in conjunction with each other, sender-verify cannot verify a sender that does not exist, mailman does not create an alias for mailman-bounces, so essentially unless one of the two is changed (which sender verify cannot) the solution is either to disable sender verify or create an alias/forwarder/account so the user does exist. My suggestion to prevent this from occurring would be to disable sender-verify in this instance since the way that mailman works is incompatible with the configuration.

This is simply misleading at best. As Steve points out, Mailman doesn't create aliases period except for the special case of MTA = 'Postfix' which does not apply to cPanel because their installations use Exim.

The issue is in cPanel's exim router for Mailman addresses. There is good documentation for creating an Exim configuration for Mailman at https://www.exim.org/howto/mailman21.html and in particular, a Mailman router at https://www.exim.org/howto/mailman21.html#roconf and transport at https://www.exim.org/howto/mailman21.html#taconf.

The problem is cPanel's Exim router and transport is different because in a 'normal' Mailman install, mail to list@example.com is piped to 'the_mail_wrapper post list' and mail to, e.g., list-bounces@example.com is piped to 'the_mail_wrapper bounces list', but in cPanel, mail to list@example.com is piped to 'the_mail_wrapper post list_example.com' and mail to, e.g., list-bounces@example.com is piped to 'the_mail_wrapper bounces list_example.com'

Now in cPanel as in all Mailman. there is a 'mailman' list so mail to all the mailman(-*) addresses should be delivered to that list, but cPanel has not programmed their Exim router/transport to know that 'mailman(-*)@example.com is a special case for which the list name is (probably I think) 'mailman' and not mailman_example.com.

If spmeone from cPanel (Lauren N or anyone) would contact me about this, I would be willing to work with them to figure out a solution, but it is not an issue in upstream Mailman, it is an issue in cPanel's Exim configuration for Mailman lists that doesn't take into account the fact the due to their changes, the 'mailman' site list is different from other lists.

...

So response #1 makes it look as if it's Mailman's fault, not cPanel's, so cPanel don't need to fix it, which begs the question: how was all this working fine until earlier this year, and what's changed that broke it?

Requiring sender-verify in Exim.

I don't know if a new version of Mailman was installed on the server in that time, but even if it were, it doesn't sound to me as if it would have changed in this department. And I think the webhost probably has had sender_verify turned on for years now, but I'm checking that with them.

If so, then I don't know.

Then response #2 makes it look as if cPanel *may* be willing to deal with the issue, despite the above.

If they would open a dialog with me, we could fix this.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/24/2017 06:06 AM, Jim Popovitch wrote:

I'm jumping in late here.. Is the problem possibly DMARC alignment failures? If so, I've been working on a patch for this:

No. This has nothing to do with DMARC. However your branch might be relevant.

The issue is that owner notifications from Mailman are sent with envelope from the sitelist-bounces address in the list's domain. In cPanel this results in the envelope being from mailman-bounces@list.domain.

cPanel's Exim configuration apparently only sees this as a valid address if a list named mailman_list.domain exists which it doesn't. There is only one 'mailman' list and its name is maybe just 'mailman' or maybe 'mailman_some_canonical_host_name', I'm not sure which but probably the former, but there isn't a separate 'mailman_list.domain' list for every list domain.

So then Exim is configured to do sender verification and mailman-bounces@list.domain is not a valid address as far as Exim is concerned so these owner notifications are bounced by Exim and never sent.

The workaround that Terry and others have implemented is to make an alias for mailman-bounces@list.domain which, cPanel's opinion to the contrary notwithstanding, I believe is the correct solution as it is the only way the domain admin is going to see any real bounces.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Thanks Mark.

Sorry for the delay in responding. I've passed your offer of help on to the webhost a few mins ago.

Hoping they take it up...

BTW, how long are my emails to this list going to be moderated for?

Terry