Mark Sapiro writes:

It's hard to say, but it seems that you have list members in multiple domains all served by the same google mail MX, and google doesn't like receiving multiple recipient domains in a single SMTP transaction.

Other way around, I think. The Google MX in question seems to be Max's smarthost. I don't think any MTAs are so "smart"[1] as to try to collect MXs for all the domains they're sending to, and then send to multiple domains via any MX that serves multiple domains.

I don't know about whitelisting, but if you set

SMTP_MAX_RCPTS = 1

This will still do the trick.

Footnotes: [1] Yes, I know about qmail. Not even djb ...!

On 3/8/2013 7:06 AM, Joseph Brennan wrote:

I reported this a few months ago, as a Google Apps for Edu customer,

I am curious. As a Google Apps for Edu customer using Google servers to relay outgoing mail, do you see the From: header and envelope sender rewriting I report at <http://mail.python.org/pipermail/mailman-users/2013-February/074748.html>?

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Joseph Brennan writes:

I reported this a few months ago, as a Google Apps for Edu customer, and Google refuses to fix it. I spent a couple of weeks back and forth with several people, and I got beyond the first line helpdesk. None of them could give me a good explanation,

I bet they think it's an anti-spam measure. None of the big services likes to talk much about that.

but they confirmed it is not considered a bug, it was done deliberately.

Can you confirm that this is happening at your outgoing MTA (ie, the first Google MX you encounter when submitting as a Google customer)?

I'm just guessing, but in practice is this issue restricted to mailing lists and other software that automatically mails to several users? Did they perhaps say "you should be using the submission protocol (port 587)"?

The reason I ask is that surely academic users regularly send mail personally to recipients at multiple domains. Some users surely have personal networks that involve multiple Google-hosted domains, and they would find their mail *very* unreliable if they were experiencing this issue. I don't see any way an institution could use such a service, or that Google could afford to not fix it. "Neither snow ... stays swift completion" and all that.

On the other hand, AFAIK queueing is rarely implemented in clients: they just hand a message text and a list of addresses to the MTA, which then decides which addresses to transfer via which MXes. The only way I can see this working at all well in the face of the issue we're discussing is that "personal" clients are subject to different authorization procedures from others. One way to accomplish that is if personal clients are using Submission to submit messages instead of SMTP. Section 9 of RFC 4409 gives specific support to having different security policies for SMTP and Submission.

Programmatic submission typically uses SMTP.

Joseph Brennan writes:

"Stephen J. Turnbull" <stephen@xemacs.org> wrote:

...

I bet they think it's an anti-spam measure. None of the big services likes to talk much about that.

That's what frontline helpdesk told me, but no one could explain how it reduces spam.

Rich man's graylisting, is my guess. Many spammers never come back for the remaining domains, I bet.

It beats me why, say, 6 recipients at one domain is less spammy than 6 recipients at two domains.

It isn't. 3 recips at one of two domains is less spammy than 6 recips at both domains, though!

We run our own smtp servers. They determine where each user's mail goes. Some users are on Google Apps, so we re-send their mail to Google's MX address. That's when the fun begins.

What I demonstrated is what happens when any host anywhere connects to Google's MX address and tries to send to addresses at two domains both hosted by Google.

But I don't understand why a host would do that in the course of ordinary mail processing, unless it is using Google's MX as a smarthost. Are you really buying anything by

(1) Looking up the MXes for all recips (2) Grouping by MX (3) Batching to each MX in the list

?

Smarthost is a special case, because the assumption is that the sender doesn't know how to route, and all outgoing mail goes through the smarthost. But the smarthost case is what the submission protocol is designed to address (and it requires authentication).

Mark Sapiro writes:

It's hard to say, but it seems that you have list members in multiple domains all served by the same google mail MX, and google doesn't like receiving multiple recipient domains in a single SMTP transaction.

Other way around, I think. The Google MX in question seems to be Max's smarthost. I don't think any MTAs are so "smart"[1] as to try to collect MXs for all the domains they're sending to, and then send to multiple domains via any MX that serves multiple domains.

I don't know about whitelisting, but if you set

SMTP_MAX_RCPTS = 1

This will still do the trick.

Footnotes: [1] Yes, I know about qmail. Not even djb ...!

On 3/8/2013 7:06 AM, Joseph Brennan wrote:

I reported this a few months ago, as a Google Apps for Edu customer,

I am curious. As a Google Apps for Edu customer using Google servers to relay outgoing mail, do you see the From: header and envelope sender rewriting I report at <http://mail.python.org/pipermail/mailman-users/2013-February/074748.html>?

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Joseph Brennan writes:

I reported this a few months ago, as a Google Apps for Edu customer, and Google refuses to fix it. I spent a couple of weeks back and forth with several people, and I got beyond the first line helpdesk. None of them could give me a good explanation,

I bet they think it's an anti-spam measure. None of the big services likes to talk much about that.

but they confirmed it is not considered a bug, it was done deliberately.

Can you confirm that this is happening at your outgoing MTA (ie, the first Google MX you encounter when submitting as a Google customer)?

I'm just guessing, but in practice is this issue restricted to mailing lists and other software that automatically mails to several users? Did they perhaps say "you should be using the submission protocol (port 587)"?

The reason I ask is that surely academic users regularly send mail personally to recipients at multiple domains. Some users surely have personal networks that involve multiple Google-hosted domains, and they would find their mail *very* unreliable if they were experiencing this issue. I don't see any way an institution could use such a service, or that Google could afford to not fix it. "Neither snow ... stays swift completion" and all that.

On the other hand, AFAIK queueing is rarely implemented in clients: they just hand a message text and a list of addresses to the MTA, which then decides which addresses to transfer via which MXes. The only way I can see this working at all well in the face of the issue we're discussing is that "personal" clients are subject to different authorization procedures from others. One way to accomplish that is if personal clients are using Submission to submit messages instead of SMTP. Section 9 of RFC 4409 gives specific support to having different security policies for SMTP and Submission.

Programmatic submission typically uses SMTP.

Joseph Brennan writes:

"Stephen J. Turnbull" <stephen@xemacs.org> wrote:

...

I bet they think it's an anti-spam measure. None of the big services likes to talk much about that.

That's what frontline helpdesk told me, but no one could explain how it reduces spam.

Rich man's graylisting, is my guess. Many spammers never come back for the remaining domains, I bet.

It beats me why, say, 6 recipients at one domain is less spammy than 6 recipients at two domains.

It isn't. 3 recips at one of two domains is less spammy than 6 recips at both domains, though!

We run our own smtp servers. They determine where each user's mail goes. Some users are on Google Apps, so we re-send their mail to Google's MX address. That's when the fun begins.

What I demonstrated is what happens when any host anywhere connects to Google's MX address and tries to send to addresses at two domains both hosted by Google.

But I don't understand why a host would do that in the course of ordinary mail processing, unless it is using Google's MX as a smarthost. Are you really buying anything by

(1) Looking up the MXes for all recips (2) Grouping by MX (3) Batching to each MX in the list

?

Smarthost is a special case, because the assumption is that the sender doesn't know how to route, and all outgoing mail goes through the smarthost. But the smarthost case is what the submission protocol is designed to address (and it requires authentication).