There is a problem with Topic regexps.
The regexp entry is compiled in re.VERBOSE mode. This can be good, but
also causes problems as the 'help' doesn't mention this. Further, the
description says 'Topic keywords, one per line, to match against each
message.' The implication is if I put
one
two
three
in the regexp box, that this topic will match keywords 'one', 'two' or
'three', but actually it matches only 'onetwothree'.
I see several ways to address this.
1) change the processing of this field to effectively join the lines
with '|' - presumably this will break existing multiline entries, but
possibly they (at least ones which already contain '|') can be
converted.
2) do 1) as a list or mm_cfg option which defaults to not doing it.
3) change the description and help for the regexp to correctly reflect
what happens.
What do others think?
--
Mark Sapiro <msapiro(a)value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
Mailman: Proposed sender-based passwords
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: this is based on Mailman 2.1.8 which, at the time of writing, is the
current version.
A discussion "approval password linked to sender rather than list?" on
mailman-users around 15 May 2006 agreed on the desirability of a variant
of the "Approved: password" mechanism, whose permission-granting password
would be associated with the email's sender rather than with the list.
The motivation is large sites with many "announcement" lists (in our case
over 2,000), where the entire membership of each such list is moderated,
and where several legitimate senders might regularly need to post single
emails to several lists. These lists might well exist in clusters.
But the option of allowing unmoderated access for those legitimate senders
opens up a major problem with spoofed email. So everyone needs to be
moderated.
But then the potentially useful moderation-bypass "Approved: listpw"
mechanism has the problems of requiring many people to share a particular
list's password, and of the passwords across a cluster of lists having to
be common. This "shared secret" problem grows even more alarmingly if the
list clusters overlap which would force reduction to a single password
across many lists, and widespread knowledge (hence leakage potential) of
that password.
Introducing an "Approval:"-like mechanism, whose password is associated
with the sender ("From:"), is intended address all this.
Virtual host considerations
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Mailman is beginning to allow some virtual hosting (i.e. multiple domains
("...@dom1", "...@dom2"). Although this is currently limited (cannot have
the same listname at multiple domains) it is clearly on the roadmap for
the future. Accordingly this password proposal attempts to take full
account of that direction.
New "sender" role
~~~~~~~~~~~~~~~~~
Currently, according to "Mailman/SecurityManager.py":
# There are current 5 roles defined in Mailman, [...]
# user, list-creator, list-moderator, list-admin, site-admin.
That is, five contexts in which passwords are used and supported.
Clarification: "user" means the intersection of a particular email address
and a particular list: the (in)famous "monthly password reminder" entity.
Put another way: the same person (email address) on 'n' different lists
is actually 'n' unrelated "users".
A new "sender" role is proposed, available across all lists at all virtual
hosts. It is implemeted as a site-wide database of email addresses and
their passwords, with the same variety of options for encryption and
storage as in (or potentially in) the other databases. Across virtual
hosts it similarly remains the same, common, sender.
Clarification: Unlike a "user", this is a single stored-once entity,
representing a single email address across the entire installation.
In each list a new list-admin function maintains a set of email addresses,
herein called "authorised_senders", that can post through the list. On
the GUI, this would present a little like various "*_these_nonmembers".
An incoming email can specify "Authorised: sender-password" in the manner
of the existing "Approved: list-password" mechanism (headers, first real
blank-separated line, etc.).
[Note: I have chosen a separate word "Authorised:". Perhaps we can
conflate this new concept onto "Approved:" if there would be no loss of
functionality. If so, then "authorised_senders" should perhaps be called
"approved_senders".]
Its processing confirms the "From:" in the list's "authorised_senders",
then does an authorisation check against the site-wide "sender" (role)
database. (If we conflate "Authorised:" and "Approved:", the password
check would potentially need to be done on both the list and the "sender"
databases.)
If an incoming "Authorised:" email is destined for several lists, perhaps
at several virtual hosts, its "From:" is processed within each list as
described above against the site-wide "sender" database.
Maintenance of "sender"
~~~~~~~~~~~~~~~~~~~~~~~
The new "sender" database requires maintenance.
Insertion: How are email addresses and their passwords added? How is it
requested? By whom or by what is it implemented? What should be done to
detect and prevent malicious (spam-like, DOS-like) mass sign-up?
Deletion: How can an email address be removed from the database? When it
is removed, how is that reflected back into all the lists which had it in
their "authorised_senders" table? (Analogous to removing a file in a
filesystem but leaving behind dangling symbolic links.) Should those
lists be left alone (but now containing redundant entries) or should their
tables be amended? Suppose the deletion from "senders" was accidental or
transient: how can any removed entries in the several "authorised_senders"
tables be efficently restored?
Changes: How does someone maintain (change) their email address and
password?
If a person changes their email address how is that reflected back to the
lists? (See "Deletion".) Intuitively, the database key would have been
the email address. But perhaps the real key should rather be some opaque,
unique (UUID-like?) entity, with the email address simply being an
attribute.
--
: David Lee I.T. Service :
: Senior Systems Programmer Computer Centre :
: Durham University :
: http://www.dur.ac.uk/t.d.lee/ South Road :
: Durham DH1 3LE :
: Phone: +44 191 334 2752 U.K. :
In response to a question on mailman-users, I looked at AvoidDups.py,
and I see that if an address is in the Cc: header of a post and is a
candidate recipient and is a list member with DontReceiveDupes set,
the address is removed from the recipient list and from the Cc: header
of the post.
I understand why it is removed from the recipient list - that's the
whole point - but does anyone know/remember why it is also removed
from the Cc: header? This seems unnecessary and inconsistent since it
isn't removed from To:, Resent-To: or Resent-Cc:.
Also, there is a docstring in the module that refers to an in memory
Message-Id: list for avoiding cross-posted dups which I think was in
the original patch, but was never incorporated. Perhaps this should be
rewritten.
--
Mark Sapiro <msapiro(a)value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
On 4/24/06, Barry Warsaw <barry(a)python.org> wrote:
> On Mon, 2006-04-24 at 16:21 +0300, Heikki Lampén wrote:
> > I'm wondering about this "true virtual hosting" issue and what is
> > going to happen to it.
> >
> > I see there's some talk about Google SoC in the wiki [0].
> > Unfortunately I don't see Mailman in the Google SoC participant list
> > [1]. Has Mailman applied as a mentoring organization? The deadline is
> > 1st of May.
>
> True, although I was thinking that if someone wants to do this, we could
> do it under the auspices of the Python Software Foundation. There
> really is no "Mailman organization" and the latter's SoC project page
> does list Mailman as a possibility.
Sounds good.
> > Anyway, it'd be awesome if SoC could finish what the vhost branch has
> > began. Or is it totally out of the question that such a patch would be
> > integrated to 2.1? Even as a branch (that'd be kept up to date with
> > 2.1) it could be very useful.
>
> It's not out of the question, although no way can it go into the
> mainline 2.1 branch. Ideally, the work would be done on the Mailman
> trunk for 2.2, although I wouldn't be opposed to an "unofficial" 2.1
> patch branch if the volunteers were so motivated. It's too big of
> project to go into 2.1 proper though.
Fair enough.
> > Why or why not should the vhost branch development happen on the same
> > subversion repository? In CVS there was the awful branching problem. I
> > think Subversion is the most attracting version control system. Then
> > again what I think doesn't matter but what you guys think is what
> > matters. Of course moving off to another platform is always extra work
> > and has to be motivated properly.
>
> My hesitance is much more the code disruption, upgrade testing, and
> support costs involved in pulling this into 2.1. We just can't be
> adding new features to 2.1 at this point. What we /really/ need to do
> is get 2.2 moving so that we can release it fairly soon (this year, I
> hope).
>
> > Also the usage of JIRA and Confluence should make it even more
> > interesting as organizing the project becomes a whole lot easier that
> > way. Maybe add a project for the vhost branch.
> >
> > Maybe I'm just being impatient but as I see it, this vhost stuff
> > should really interest a lot of people.
>
> Definitely. Everyone wants to see it, and it will definitely happen for
> 2.2. It would be awesome if a SoC volunteer wanted to take it on and I
> would be more than happy to mentor the work. Are you a student and are
> you interested in working on it?
I'm unfortunately already employed during summer time (would've had
been an interesting project). Maybe someone on the list is or knows a
promising victim?
--
heze
heikki.lampen(a)gmail.com
Per Barry's request, I thought I'd describe some of the issues our
sites has encountered with Mailman lists and bounce messages.
At our site, we currently 15,775 subscribers (8,296 unique) across 549
lists. While a few of the lists are huge (hundreds or thousands of
subscribers), the median list size is only 4 subscribers.
The primary issue we are encountering is that the vast majority of our
lists (even the ones with hundreds of subscribers) are
announcement-type lists, not discussion-type lists. The list owners
are using the lists to reach coworkers, specific customers, course
attendees, etc. The list admins know exactly which addresses are on
their lists, because they've added them all themselves.
Situations like this happen all the time:
1. A VIP asks an administrative assistant to send out a message
to a group of people: team members, specific customers, etc.
2. The AA sends the message to the appropriate Mailman list.
Unbeknown to the AA, one of the addresses bounces.
3. The person whose address bounced doesn't get the message, and
(e.g.) misses an important meeting, fails to provide some
important information, etc.
4. The VIP comes down hard on the AA for failing to notify
everyone.
5. The AA comes to us full a fire and brimstone, complaining the
Mailman doesn't permit people to perform necessary business
functions.
The issue is that people in this situation *want* to see the
individual bounces, and they want to see them *immediately*, because
the bounces contain important information that they might need to act
upon. The closest they can get to that right now is to configure the
list so that a single bounce immediately removes the subscriber from
the list. But this isn't always the correct thing to do. (For
example, lots of people seem to have broken vacation programs that
respond to *everything*, and they continually get unsubscribed from
lists.)
The more I think about this, the more I think that silently discarding
bounces is an error. While some bounces are irrelevant (e.g. vacation
messages), in the majority of cases, bounces are information that some
entity should act upon. There should be two (and only two) options:
1. Mailman processes all bounces.
2. All bounces are forwarded to the list administrator(s).
I've attached my first attempt at implementing this behavior. (I say
"first attempt" because I'd really like to get both the listname and
the address that bounced into the Subject header, but to do that
requires more complicated logic than what I've implemented so far.)
If list admins *really* want to (effectively) discard all bounces,
they can e.g. set bounce_info_stale_after to 1 and then set
bounce_score_threshold to something impossibly high (e.g. 500,000).
Thoughts?
James
On 2006-04-27 at 22:46-05 Brad Knowles <brad(a)stop.mail-abuse.org> wrote:
> At 10:40 AM -0500 2006-04-27, Neal Groothuis wrote:
>
> > Again from RFC 2822 3.6.2, the Sender: header should contain the
> > address of the agent responsible for transmitting the message,
> > meaning that a person who sends mail to the address in that header
> > should expect to reach said agent, not suggest to Mailman that a
> > message bounced.
>
> Right. Mailman is the agent responsible for transmitting the
> message, and this needs to be reflected.
This is not correct. Quoting RFC2822 section 3.6.2:
3.6.2. Originator fields
The originator fields indicate the mailbox(es) of the source of
the message. The "From:" field specifies the author(s) of the
message, that is, the mailbox(es) of the person(s) or system(s)
responsible for the writing of the message. The "Sender:" field
specifies the mailbox of the agent responsible for the actual
transmission of the message. For example, if a secretary were to
send a message for another person, the mailbox of the secretary
would appear in the "Sender:" field and the mailbox of the actual
author would appear in the "From:" field.
The Sender header should be employed by the orignator of the message,
and only the originator. Mailman is not the originator of a message
sent to a list; it is merely a relay agent.
I will grant that the phrasing of the RFC is suboptimal here--it uses
"transmission" when a better word choice would have been "submission".
But the immediately proceeding example (of a secretary sending mail on
behalf of another person) clarifies the intent beyond any claim of
ambiguity.
> [Outlook's behavior] is an MUA problem. See FAQ 2.3.
No, it's not. As much as it pains me to say it, Outlook's behavior
matches perfectly the intent of the RFC. It is Mailman's behavior of
rewriting the Sender header that is the problem.
> However, we want to make sure to capture any potential messages that
> may be routed to the "Sender:" field and have them automatically
> processed through the part of the system that is designed to do that
> sort of thing.
Mailman's "processing" behavior is to treat a reply to the Sender as a
bounce. This is incorrect behavior, because many mail clients will
include address of the Sender header in a "reply-to-all" function,
causing Mailman to treat the reply as a bounce.
So, in summary, the disadvantages of Mailman's behavior of rewriting
the Sender header is that doing so is not in the intended spirit of
RFC2822, causes subscription grief, and breaks Outlook. The advantage
is that it helps Mailman detect bounces from a slim minority of
brain-dead MTAs that send bounces to the Sender header.
> The problem is that you said you wanted to implement an option to
> allow people to turn it off, not to rip this feature completely out
> of the system.
I would argue that the best course of action is to excise Sender
header rewriting entirely and provide no option to turn it on.
(Mailman has way too many options already.)
If, however, an option is created to control the behavior, it should
definitely default to OFF (no Sender header rewriting), not on.
--
James Ralston, Information Technology
Software Engineering Institute
Carnegie Mellon University, Pittsburgh, PA, USA
Hello,
can someone point me to the procedure/documents needed to assign copyright
to Barry^H^H^H^H^H the FSF ? I can't find them on Google or on the GNU
project pages (??).
-- Fil
hello,
I am wondering why any future template suggestion a priori rule out
PageTemplates? I find them really handy to use, at least in zope-contexts.
Is it becuase it is zope or other reasons?
thanks,
/dario
--
-- -------------------------------------------------------------------
Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
Lyrics applied to programming & application design:
"emancipate yourself from mental slavery" - redemption song, b. marley
Hi,
I think our new logger should set group writable to the new logs. I
tried a fresh install and get cgi error after mailmanctl start /
mailmanctl errot after cgi execution, vice versa.
The patch should look like this:
Index: Mailman/loginit.py
===================================================================
--- Mailman/loginit.py (revision 7892)
+++ Mailman/loginit.py (working copy)
@@ -103,6 +103,8 @@
# There was also a 'debug' logger, but that was mostly unused, so
instead
# we'll use debug level on existing loggers.
#
+ # First set umask because both mailman/www write logs
+ oumask = os.umask(002)
# Start by creating a common formatter and the root logger.
formatter = logging.Formatter(fmt=FMT, datefmt=DATEFMT)
log = logging.getLogger('mailman')
@@ -120,6 +122,8 @@
_handlers.append(handler)
handler.setFormatter(formatter)
log.addHandler(handler)
+ # restore original umask
+ os.umask(oumask)
--
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/
I think I've finally gotten a grasp of the new Mailman logging scheme
and I think there's something missing.
I see how the basic logging to files works and how the propagate flag
to loginit.initialize() controls propagation to the root 'mailman'
logger which logs to the sys.stderr stream.
What I think is missing is something analogous to the old
Mailman.Logging.Utils.LogStdErr() to cause writes to sys.stderr to be
logged instead of or in addition to being sent to the stderr stream.
The end result of this is that a few messages from bin/mailmanctl and
one message from bin/qrunner will only go to the stderr stream and
will not be logged. I don't see this as serious as I think all these
messages can only occur in response to fatal conditions in direct
response to a mailmanctl command.
However, there are also sys.stderr writes in the various scripts
invoked by the mail wrapper. These used to be logged as well as
written to the stderr stream (where they presumably end up in a DSN).
The error conditions that cause these messages occur because of bad
alaises or routers in the MTA (things that invoke the wrapper without
a list name) or aliases left after a list is removed. These should
probably still be logged.
--
Mark Sapiro <msapiro(a)value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
