Re: [Mailman-Users] AOL redacts user addresses even with VERP and full personalization enabled
I'm surprised to read in this thread that the terms of service for AOL's feedback loop forbid us from using its reports to identify users.
The page where you sign up for the FBL seems to say just the opposite (end of third paragraph):
"We suggest using opaque identifiers for the email recipient or a custom remove link in the body of the email to help you identify the original recipient of the message."
http://postmaster.aol.com/Postmaster.FeedbackLoop.php
rac
On Jun 19, 2012, at 5:17 PM, Russell Clemings wrote:
I'm surprised to read in this thread that the terms of service for AOL's feedback loop forbid us from using its reports to identify users.
The page where you sign up for the FBL seems to say just the opposite (end of third paragraph):
"We suggest using opaque identifiers for the email recipient or a custom remove link in the body of the email to help you identify the original recipient of the message."
I haven't read the page in question in the last few years, but assuming what you've quoted is accurate then this would be a "recent development" in the life of the AOL FBL as I know it.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
On Tue, Jun 19, 2012 at 8:17 PM, Russell Clemings <rclemings@gmail.com>wrote:
I'm surprised to read in this thread that the terms of service for AOL's feedback loop forbid us from using its reports to identify users.
The page where you sign up for the FBL seems to say just the opposite (end of third paragraph):
"We suggest using opaque identifiers for the email recipient or a custom remove link in the body of the email to help you identify the original recipient of the message."
Way to embarrass all of us who didn't take the time to actually read the TOS! ;-)
Now that you motivated me, I actually read the blog post too: http://postmaster-blog.aol.com/2008/08/13/more-on-the-upcoming-feedback-loop...
It now seems pretty clear that we can use these reports in the way Lindsay and others have proposed.
Thanks!
On Tue, 2012-06-19 at 20:42 -0400, Dave (FitEyes) wrote:
Now that you motivated me, I actually read the blog post too: http://postmaster-blog.aol.com/2008/08/13/more-on-the-upcoming-feedback-loop...
It now seems pretty clear that we can use these reports in the way Lindsay and others have proposed.
Well if this is so, are they still redacting the VERP recipient addresses?
-- Lindsay Haisley | "We are all broken toasters, but we still FMP Computer Services | manage to make toast" 512-259-1190 | http://www.fmp.com | - Cheryl Dehut
On Tue, 2012-06-19 at 20:42 -0400, Dave (FitEyes) wrote:
Now that you motivated me, I actually read the blog post too: http://postmaster-blog.aol.com/2008/08/13/more-on-the-upcoming-feedback-loop...
It now seems pretty clear that we can use these reports in the way Lindsay and others have proposed.
Well if this is so, are they still redacting the VERP recipient addresses?
-- Lindsay Haisley | "We are all broken toasters, but we still FMP Computer Services | manage to make toast" 512-259-1190 | http://www.fmp.com | - Cheryl Dehut
On Tue, Jun 19, 2012 at 8:55 PM, Lindsay Haisley <fmouse-mailman@fmp.com>wrote:
On Tue, 2012-06-19 at 20:42 -0400, Dave (FitEyes) wrote:
Now that you motivated me, I actually read the blog post too:
http://postmaster-blog.aol.com/2008/08/13/more-on-the-upcoming-feedback-loop...
It now seems pretty clear that we can use these reports in the way Lindsay and others have proposed.
Well if this is so, are they still redacting the VERP recipient addresses?
Yes.
On Tue, 2012-06-19 at 21:07 -0400, Dave (FitEyes) wrote:
On Tue, Jun 19, 2012 at 8:55 PM, Lindsay Haisley <fmouse-mailman@fmp.com> wrote: On Tue, 2012-06-19 at 20:42 -0400, Dave (FitEyes) wrote: > It now seems pretty clear that we can use these reports in the way > Lindsay and others have proposed.
Well if this is so, are they still redacting the VERP recipient addresses?Yes.
Reading in <http://postmaster.aol.com/Postmaster.FeedbackLoop.php+> it seems that "[AOL suggests] using opaque identifiers for the email recipient or a custom remove link in the body of the email to help you identify the original recipient of the message."
I would assume that an AES-encrypted email address in Resent-Message-ID or in the VERP address, or even a hashed recipient address in a custom header such as X-Subdata, all of which have been discussed here, would meet the criterion of being an "opaque identifier". Does this sound logical?
Of these, an encrypted or hashed recip address in the VERP envelope header seems the most logical, since it seems that we don't have to go "stealth" with this one. Any chance of requesting this in Mailman 3?
Looking at a recent Email Feedback Report it looks as if the list name is also pretty well redacted, except in the message footer, the format of which is up to individual list administrators, so maybe the list name or address should be included in this or another encrypted header as well.
-- Lindsay Haisley | "The voice of dissent was arrested before FMP Computer Services | the president cleared his throat to 512-259-1190 | speak of freedom" http://www.fmp.com | | -- Chris Chandler
Lindsay Haisley writes:
Any chance of requesting this in Mailman 3?
As usual, the advice is to file a bug report/RFE on Launchpad, Mailman project, tag it Mailman 3 (or maybe that's milestone Mailman 3?)
If you want more discussion from the core people (well, Barry; Mark's presumably already said everything he wants to say about this subject :-), you could send mail to mailman-developers, but I think this idea is already pretty well-baked, and maybe you even have a patch you could attach to the issue?
On Wed, 2012-06-20 at 14:39 +0900, Stephen J. Turnbull wrote:
Lindsay Haisley writes:
Any chance of requesting this in Mailman 3?
As usual, the advice is to file a bug report/RFE on Launchpad, Mailman project, tag it Mailman 3 (or maybe that's milestone Mailman 3?)
If you want more discussion from the core people (well, Barry; Mark's presumably already said everything he wants to say about this subject :-), you could send mail to mailman-developers, but I think this idea is already pretty well-baked, and maybe you even have a patch you could attach to the issue?
I was thinking of posting to the dev list, to which I also subscribe, and inquiring with regard to the advisability of putting this, as you suggested, into the Resent-Message-ID header, as opposed to the VERP address or some custom header.
My thinking, from corresponding with Dave and my own observation, is that both the list address and the recipient address should be AES encrypted and passed in a single header, and because this information is pretty much guaranteed to be unique per message, given that my AES encryption routine uses random input, using the Resent-Message-ID header would fulfill a dual purpose and satisfy RFC 2822. The use of this header would depend on whether the current v3 development blueprint has plans for this header which would preempt its use for this purpose.
I posted code and patches earlier on this list, but the patch is against Mailman 2.1.15 rather than Mailman 3, which is the current development focus. I imagine it's rather different. I'd have to take a look at the code and figure out where the patch might go.
I'm also not up on what the execution time hit would be in generating a short AES cipher for each outgoing message. This might be considerable on a large list with many thousands of subscribers. As it is now, in my patch, if VERP is not enabled, or there is no personalization, which I believe excludes VERP, then no encrypted recipient cipher would be generated.
When I get a chance I'll take a look at the v3 code.
Lindsay Haisley writes:
I posted code and patches earlier on this list, but the patch is against Mailman 2.1.15 rather than Mailman 3, which is the current development focus. I imagine it's rather different.
The code is organized quite differently, but I suspect that the handler architecture will look familiar to you.
From the reports I've received, it looks as if they redact only from the headers. With personalization on, I put a "%(user_address)s" token in the non-digest footer and as of the last report I got (June 8) it came through the feedback loop intact. I've never figured out a similar fix for digests, however, and that seems to be where most of the reports come from. So maybe there's room for a new approach there.
rac
---------- Forwarded message ----------
From: Lindsay Haisley <fmouse@fmp.com> To: mailman-users@python.org Cc: Date: Tue, 19 Jun 2012 19:54:34 -0500 Subject: Re: [Mailman-Users] AOL redacts user addresses even with VERP and full personalization enabled On Tue, 2012-06-19 at 20:42 -0400, Dave (FitEyes) wrote:
Now that you motivated me, I actually read the blog post too:
http://postmaster-blog.aol.com/2008/08/13/more-on-the-upcoming-feedback-loop...
It now seems pretty clear that we can use these reports in the way Lindsay and others have proposed.
Well if this is so, are they still redacting the VERP recipient addresses?
-- Lindsay Haisley | "We are all broken toasters, but we still FMP Computer Services | manage to make toast" 512-259-1190 | http://www.fmp.com | - Cheryl Dehut
On Wed, 2012-06-20 at 18:23 -0700, Russell Clemings wrote:
From the reports I've received, it looks as if they redact only from the headers. With personalization on, I put a "%(user_address)s" token in the non-digest footer and as of the last report I got (June 8) it came through the feedback loop intact.
This may be true, however relying on cleartext in the footer information to identify the recipient has two problems.
First, it restricts the freedom of the list administrator to put whatever he/she wants in the footer, and because the form of footer information is friable, depending on the list admin, it's impossible to write a one-size-fits-all script to pull subscriber addresses from Feedback Reports and deal with complaining subscribers. Putting this information in a header which is added depending only on whether personalization/verp is enabled or not is independent of what the list admin decides he/she wants subscribers to see in the footer - which should be there for the benefit of subscribers, not list admins.
Second, putting the subscriber's email address as cleartext in _any_ part of a post makes it subject to AOL's redaction process. Whether or not they are currently redacting this in footer information doesn't mitigate the fact that they reserve the right to do so, according to their TOS. Changes to what is and isn't redacted over the past couple of years indicates that they periodically change or refine this process. It seems, however, according to their online documents, that if the recipient address is encrypted or hashed, then it meets their spec and won't raise objections, or redactions.
I've never figured out a similar fix for digests, however, and that seems to be where most of the reports come from. So maybe there's room for a new approach there.
If the information in in the header, it's there regardless of whether a subscriber chooses to receive digests or individual posts.
-- Lindsay Haisley | "The only unchanging certainty FMP Computer Services | is the certainty of change" 512-259-1190 | http://www.fmp.com | - Ancient wisdom, all cultures
participants (6)
-
Brad Knowles -
Dave (FitEyes) -
Lindsay Haisley -
Lindsay Haisley -
Russell Clemings -
Stephen J. Turnbull