Mailman 2.1.10b4 Released
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I am happy to announce the next beta release of Mailman 2.1.10.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support for three new language translations, Galician, Hebrew and Slovak and a few new features.
Mailman is free software for managing email mailing lists and e- newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites.
For more information, including download links, please see:
http://www.list.org http://mailman.sf.net http://www.gnu.org/software/mailman
Special thanks are due to Barry Warsaw and Tokio Kikuchi for much coding and support, Moritz Naumann for help with security issues and Jim Tittsler for a significant patch.
Here's a list of the major changes.
Security
- The 2.1.9 fixes for CVE-2006-3636 were not complete. In particular, some potential cross-site scripting attacks were not detected in editing templates and updating the list's info attribute via the web admin interface. This has been assigned CVE-2008-0564 and has been
fixed. Thanks again to Moritz Naumann for assistance with this.
New Features
- Changed cmd_who.py to list all members if authorization is with the list's admin or moderator password and to accept the password if the roster is public. Also changed the web roster to show hidden members when authorization is by site or list's admin or moderator password (1587651).
- Added the ability to put a list name in accept_these_nonmembers to accept posts from members of that list (1220144).
- Added a new 'sibling list' feature to exclude members of another list from receiving a post from this list if the other list is in the To: or Cc: of the post or to include members of the other list if that list is not in the To: or Cc: of the post (Patch ID 1347962).
- Added the admin_member_chunksize attribute to the admin General Options interface (Bug 1072002, Partial RFE 782436).
Internationalization
- Added the Hebrew translation from Dov Zamir. This includes addition of a direction ('ltr', 'rtl') to the LC_DESCRIPTIONS table. The add_language() function defaults direction to 'ltr' to not break existing mm_cfg.py files.
- Added the Slovak translation from Martin Matuska.
- Added the Galician translation from Frco. Javier Rial RodrÃguez.
Changes since 2.1.10b3 include the Galician translation and updates to the French translation (Vietnamese and Danish translations were updated in 2.1.10b3). Other changes since 2.1.10b3 include:
- In 2.1.9, queue runner processing was made ~ more robust by making backups of queue entries when they were dequeued ~ so they could be recovered in the event of a system failure. This ~ opened the possibility that if a message itself caused a runner to ~ crash, a loop could result that would endlessly reprocess the message. ~ This has now been fixed by adding a dequeue count to the entry and ~ moving the entry aside and logging the fact after the third dequeue of ~ the same entry.
- Fixed the command line scripts add_members, sync_members and ~ clone_member to properly handle banned addresses (1904737).
- Fixed bin/newlist to add the list's preferred language to the list's ~ available_languages if it is other than the server's default language ~ (1906368).
- Changed the first URL in the RFC 2369 List-Unsubscribe: header to go ~ to the options login page instead of the listinfo page.
- Changed the options login page to not issue the "No address given" ~ error when coming from the List-Unsubscribe and other direct links. ~ Also changed to remember the user's language selection when ~ redisplaying the page following an error.
/Mark Sapiro
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFH2eJuVVuXXpU7hpMRAihOAJ4zIREWCWCQt7YDDHp3frDHjzwkCQCfdh7J W3UKWsTTfStBE4z64oqa36c= =ZedT -----END PGP SIGNATURE-----
On Thu, Mar 13, 2008 at 10:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
I am happy to announce the next beta release of Mailman 2.1.10.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version.
Everyone? Or just 2.1.10 beta testers?
Are you saying that any 2.1.9 or less Mailman systems need to be upgraded ASAP to at least 2.1.10b4?
Thx,
-Jim P.
Jim Popovitch wrote:
On Thu, Mar 13, 2008 at 10:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
I am happy to announce the next beta release of Mailman 2.1.10.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version.
Everyone? Or just 2.1.10 beta testers?
Are you saying that any 2.1.9 or less Mailman systems need to be upgraded ASAP to at least 2.1.10b4?
It may be overstated. At a minimum, everyone should be planning to upgrade to 2.1.10 final when it is available, but the 2.1.10 beta releases address a security issue that is rather obscure and difficult to exploit, but which has been publically exposed by the releases themselves, thus sites which are concerned about this issue should upgrade now and then again when the final is released.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
For translators who haven't yet updated their translations for 2.1.10 and for other interested parties, I plan to release a 2.1.10 release candidate on April 14, 2008 and barring any issues with the RC, the 2.1.10 final on April 21, 2008.
Note to translators - the latest mailman.pot for the 2.1 branch is at <https://code.launchpad.net/~mailman-coders/mailman/2.1>. The SVN repository on sourceforge is no longer maintained and is out of date.
One way to get your updated translation to me is to register on launchpad and make your own private branch off the 2.1 branch, update your branch and publish it and send me a note when it's ready. Then I can just merge your branch back into the 2.1 branch.
Of course, if you prefer, you can just send me your updated message catalog and templates if any or make them available anywhere on the web and tell me where to find them.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFH97KHVVuXXpU7hpMRAoSnAKCb9kN0WV7EVJ2KVqUX0yhPQjlkkQCgwfLx UjpxqBDIR8yLPin2faeMr7M= =yLBQ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 5, 2008, at 1:10 PM, Mark Sapiro wrote:
For translators who haven't yet updated their translations for 2.1.10 and for other interested parties, I plan to release a 2.1.10 release candidate on April 14, 2008 and barring any issues with the RC, the 2.1.10 final on April 21, 2008.
Sounds great, Mark. Thanks!
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkf3ygwACgkQ2YZpQepbvXFxGACgpa03tySXUxgGL8yuQCzD7zub QeoAmwbL7R3SKQ+1K5aF9EiS8d3z5YiM =t5NW -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I am happy to announce the release of Mailman 2.1.10rc1.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support for three new language translations, Galician, Hebrew and Slovak and a few new features.
Mailman is free software for managing email mailing lists and e- newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites.
For more information, including download links, please see:
http://www.list.org http://mailman.sf.net http://www.gnu.org/software/mailman
Special thanks are due to Barry Warsaw and Tokio Kikuchi for much coding and support, Moritz Naumann for help with security issues and Jim Tittsler for a significant patch.
Here's a list of the major changes.
Note in particular, the second item under Security as this is new since 2.1.10b4 and requires an mm_cfg.py change to maintain current behavior.
Security
- The 2.1.9 fixes for CVE-2006-3636 were not complete. In particular, ~ some potential cross-site scripting attacks were not detected in ~ editing templates and updating the list's info attribute via the web ~ admin interface. This has been assigned CVE-2008-0564 and has been ~ fixed. Thanks again to Moritz Naumann for assistance with this.
- There is a new mm_cfg.py/Defaults.py variable ~ OWNERS_CAN_CHANGE_MEMBER_PASSWORDS which controls whether the list ~ owner can change a member's password from the member's options page. ~ This defaults to No and should be changed to Yes only if list owners ~ are trusted to not change a member's password, log in as the member ~ and make global membership changes.
New Features
- Changed cmd_who.py to list all members if authorization is with the ~ list's admin or moderator password and to accept the password if the ~ roster is public. Also changed the web roster to show hidden members ~ when authorization is by site or list's admin or moderator password ~ (1587651).
- Added the ability to put a list name in accept_these_nonmembers ~ to accept posts from members of that list (1220144).
- Added a new 'sibling list' feature to exclude members of another list ~ from receiving a post from this list if the other list is in the To: ~ or Cc: of the post or to include members of the other list if that ~ list is not in the To: or Cc: of the post (Patch ID 1347962).
- Added the admin_member_chunksize attribute to the admin General ~ Options interface (Bug 1072002, Partial RFE 782436).
Internationalization
- Added the Hebrew translation from Dov Zamir. This includes addition ~ of a direction ('ltr', 'rtl') to the LC_DESCRIPTIONS table. The ~ add_language() function defaults direction to 'ltr' to not break ~ existing mm_cfg.py files.
- Added the Slovak translation from Martin Matuska.
- Added the Galician translation from Frco. Javier Rial RodrÃguez.
Changes since 2.1.10b4 include the OWNERS_CAN_CHANGE_MEMBER_PASSWORDS setting mentioned above plus
- Changed cmd_subscribe.py to properly accept (no)digest without a ~ password and to recognize (no)digest and address= case insensitively.
- An updated mm-handler (mm-handler-2.1.10) that can help reduce ~ backscatter has been added to the contrib directory.
and updates to the Italian and Polish translations.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFIA98ZVVuXXpU7hpMRAm3uAKCngufNpjWZxTxIupg2X1dd5qSbLACgsAQX xchWm2WMfDzXET53TeLxJcw= =ZT1m -----END PGP SIGNATURE-----
On Mon, 14 Apr 2008, Mark Sapiro wrote:
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support
Quick clarification: It's somewhat unusuall for it to be recommended that all sites upgrade to a release candidate. Am I to understand that the recommendation is for sites running 2.1.9 release updates to the 2.1.10rc1 -- or just those running previous 2.1.10 betas ?
========================================================== Chris Candreva -- chris@westnet.com -- (914) 948-3162 WestNet Internet Services of Westchester http://www.westnet.com/
Christopher X. Candreva wrote:
On Mon, 14 Apr 2008, Mark Sapiro wrote:
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support
Quick clarification: It's somewhat unusuall for it to be recommended that all sites upgrade to a release candidate. Am I to understand that the recommendation is for sites running 2.1.9 release updates to the 2.1.10rc1 -- or just those running previous 2.1.10 betas ?
There are two security issues mentioned in the announcement. The first of these was exposed and addressed in 2.1.10b1 and the second in 2.1.10rc1. That is the reason for the recommendation. However, per the previously announced schedule, the 2.1.10 final release is expected to be available on April 21, so if you've waited this long, you might as well wait for the final.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Apr 15, 2008 at 11:56 AM, Mark Sapiro <mark@msapiro.net> wrote:
There are two security issues mentioned in the announcement.
<harsh criticism> How much sense does it make to announce security issues in a release CANDIDATE? Come on guys, release a STABLE version (or FIX), then announce. <--- Standard Operation Procedures. </harsh criticism>
Quit feeding the enemy instead of your supporters.
Thx,
-Jim P.
On Tue, April 15, 2008 16:24, Jim Popovitch wrote:
On Tue, Apr 15, 2008 at 11:56 AM, Mark Sapiro <mark@msapiro.net> wrote:
There are two security issues mentioned in the announcement.
<harsh criticism> How much sense does it make to announce security issues in a release CANDIDATE? Come on guys, release a STABLE version (or FIX), then announce. <--- Standard Operation Procedures. </harsh criticism>
Quit feeding the enemy instead of your supporters.
Thx,
-Jim P.
I'm going to be harshly critical as well. Did you even read the release notes in the announcement?
You are completely off base here. While Mark did not explicitly say so in his reply, the fixes for the security problems noted are in the release. If you had read the notes, you would see that. I suggest you go back and read the original announcement in full.
I'm currently using the 2.1.10b4 release and it has been quite stable since I went to it. I have no qualms about moving to the 2.1.10rc1 release and will do so as soon as I can manage the time to do it.
My experience has been that by the time a release candidate is announced by this project, it is usually quite close to the final version and the only changes that are made in a stable release are often just translation issues.
-- Dragon
Venimus, Saltavimus, Bibimus (et naribus canium capti sumus)
On Tue, Apr 15, 2008 at 8:49 PM, Dragon <dragon@crimson-dragon.com> wrote:
I'm going to be harshly critical as well. Did you even read the release notes in the announcement?
Yes, I did.
You are completely off base here. While Mark did not explicitly say so in his reply, the fixes for the security problems noted are in the release.
You seem to think that all sites will magically dig the fix out of a RC release to apply to 2.1.9 or less install.
If you had read the notes, you would see that. I suggest you go back and read the original announcement in full.
You are assuming, incorrectly, that I never read it.
I'm currently using the 2.1.10b4 release and it has been quite stable since I went to it. I have no qualms about moving to the 2.1.10rc1 release and will do so as soon as I can manage the time to do it.
My experience has been that by the time a release candidate is announced by this project, it is usually quite close to the final version and the only changes that are made in a stable release are often just translation issues.
That is *your* experience, but that may only be unique to *you*. Quit thinking that *your* scenario works in everyone else's model.
Through out this whole 2.1.10 RC process, Mark has implied over and over that the RC (release CANDIDATE) was recommended for immediate replacement of all previous versions of Mailman. I questioned that a month ago, and was told that the release CANDIDATE contained security fixes not available to prior versions. Everyone now knows what those vulnerabilities are, but there is no *fix* for prior versions, nor is there a *stable* (i.e. non-release CANDIDATE) available with the fixes. Sure the release CANDIDATE might work in some situations, but what about those environments where only final releases, or non-development code, are permitted by policy?
-Jim P.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 15, 2008, at 8:49 PM, Dragon wrote:
My experience has been that by the time a release candidate is
announced by this project, it is usually quite close to the final version and
the only changes that are made in a stable release are often just
translation issues.
Besides, our entire source code is freely available, as is the archive
of our commit messages. We actually embargo certain commits until a
time that we and the security professionals we work with agree, but
once the changes are committed, I think you can pretty much treat them
as being exposed publicly. Better to get a release out asap after
that and let the community know that there are important fixes
contained within.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iQCVAwUBSAVSF3EjvBPtnXfVAQKc8gQAiCd3Ri8gSweRci7zknhvmoPpKEB5wlj9 Q3AQMLx2R/cCPsqPIH8mwE5cT9F7xN5VmfA/XjZwVmbfGztDJw2ZJguxGO/q03lN vTQvMo0JG/GQPyL58j0xM6INntC9LnIzzJ/5PN5AGhVH1+PaXpFpXw60bcX29IGp fkEqw7ehl60= =HS13 -----END PGP SIGNATURE-----
On Tue, Apr 15, 2008 at 9:10 PM, Barry Warsaw <barry@python.org> wrote:
Better to get a release out asap after that and let the community know that there are important fixes contained within.
Fair enough. Where's the release then?
Look, I know you folks are working hard on this, and I certainly don't dis-respect that. HOWEVER, the process flow needs some re-thinking. You should not publicly release security vulnerability details before fixes are identified for current versions. I can't imagine that you don't already know that.
-Jim P.
Quoting Jim Popovitch <yahoo@jimpop.com>:
Fair enough. Where's the release then?
Dragon is right -- the code is up-to-date and waiting for translation,
as do pretty much all RCs released by this project.
Look, I know you folks are working hard on this, and I certainly don't dis-respect that.
Actions speak louder than words, and your actions in this case tell me
that you most definitely *do* dis-respect us.
Feel free to act in some other manner that proves me wrong.
HOWEVER, the process flow needs some re-thinking.You should not publicly release security vulnerability details before fixes are identified for current versions. I can't imagine that you don't already know that.
I don't care what you think. You don't run this project. We have to
support a wide variety of languages, and we can't just release an
English-only version when we think it's ready.
The best we can do is to put out an RC and tell people what the
reasons are for wanting to upgrade to this code, so long as they know
it's only going to be available in limited numbers of languages.
Others can wait for the updated versions that support their language
of choice.
Each person installing or running Mailman gets to make their own
choice as to when they're going to pull that trigger. The security
notices are out there, there is an RC available which addresses them.
If you feel the burning need to have the latest and greatest code
which avoids the more recent security issues, then go and install the
RC.
But unless you're putting up the money to run this entire project out
of your own personal pocket, I'm tired of listening to you bitch at us
for not following your rules regarding how a release should be done.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
On Tue, Apr 15, 2008 at 9:44 PM, Brad Knowles <brad@shub-internet.org> wrote:
Quoting Jim Popovitch <yahoo@jimpop.com>:
Fair enough. Where's the release then?
Dragon is right -- the code is up-to-date and waiting for translation, as do pretty much all RCs released by this project.
Look, I know you folks are working hard on this, and I certainly don't dis-respect that.
Actions speak louder than words, and your actions in this case tell me that you most definitely *do* dis-respect us.
Feel free to act in some other manner that proves me wrong.
HOWEVER, the process flow needs some re-thinking.You should not publicly release security vulnerability details before fixes are identified for current versions. I can't imagine that you don't already know that.
I don't care what you think. You don't run this project. We have to support a wide variety of languages, and we can't just release an English-only version when we think it's ready.
The best we can do is to put out an RC and tell people what the reasons are for wanting to upgrade to this code, so long as they know it's only going to be available in limited numbers of languages. Others can wait for the updated versions that support their language of choice.
Each person installing or running Mailman gets to make their own choice as to when they're going to pull that trigger. The security notices are out there, there is an RC available which addresses them. If you feel the burning need to have the latest and greatest code which avoids the more recent security issues, then go and install the RC.
But unless you're putting up the money to run this entire project out of your own personal pocket, I'm tired of listening to you bitch at us for not following your rules regarding how a release should be done.
Nice.
Sad too.
-Jim P.
If you want to do something that is actually productive here, why
don't you find a way to use your own resources and your own personal
free time to resolve this issue?
Maybe you could run a very large mailing list server you'd be willing
to use as a guinea pig for all RC's, so that we would have an
independent certification process, whereby you willingly sign away
your first born if the code proved to have bugs in it, even after you
certified it as being perfect.
-- Brad Knowles <brad@shub-Internet.org>
Sent from my iPhone
On Apr 15, 2008, at 8:56 PM, "Jim Popovitch" <yahoo@jimpop.com> wrote:
On Tue, Apr 15, 2008 at 9:44 PM, Brad Knowles <brad@shub- internet.org> wrote:
Quoting Jim Popovitch <yahoo@jimpop.com>:
Fair enough. Where's the release then?
Dragon is right -- the code is up-to-date and waiting for
translation, as do pretty much all RCs released by this project.Look, I know you folks are working hard on this, and I certainly
don't dis-respect that.Actions speak louder than words, and your actions in this case tell
me that you most definitely *do* dis-respect us.Feel free to act in some other manner that proves me wrong.
HOWEVER, the process flow needs some re-thinking.You should not publicly release security vulnerability details
before fixes are identified for current versions. I can't imagine that
you don't already know that.I don't care what you think. You don't run this project. We have to support a wide variety of languages, and we can't just release an English-only version when we think it's ready.
The best we can do is to put out an RC and tell people what the
reasons are for wanting to upgrade to this code, so long as they know it's only
going to be available in limited numbers of languages. Others can wait for
the updated versions that support their language of choice.Each person installing or running Mailman gets to make their own
choice as to when they're going to pull that trigger. The security notices
are out there, there is an RC available which addresses them. If you feel
the burning need to have the latest and greatest code which avoids the
more recent security issues, then go and install the RC.But unless you're putting up the money to run this entire project
out of your own personal pocket, I'm tired of listening to you bitch at us
for not following your rules regarding how a release should be done.Nice.
Sad too.
-Jim P.
Jim Popovitch wrote:
Fair enough. Where's the release then?
Look, I know you folks are working hard on this, and I certainly don't dis-respect that. HOWEVER, the process flow needs some re-thinking. You should not publicly release security vulnerability details before fixes are identified for current versions. I can't imagine that you don't already know that.
I appreciate your view Jim, and I was remis in not making patches for 2.1.9 publicly announced and available[1], however, if you don't trust my 2.1.10 beta or rc release to be stable enough for production use, why would you think my patches for 2.1.9 would be any better?
I really am faced with only two choices. Commit my fixes to the publicly available source tree so they can be exposed and tested in a wide variety of environments during the beta release phase, which process necessarily also exposes the vulnerabilities that they fix to the world, or sit on my patches and release them untested by others in the final release.
[1]Patches for CVE-2008-0564 were made available to those who asked, and a google search will show that some distros have been patched, although Ubuntu for example <https://bugs.launchpad.net/ubuntu/+source/mailman/+bug/199338> calls it "low" importance.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Apr 15, 2008 at 11:04 PM, Mark Sapiro <mark@msapiro.net> wrote:
I appreciate your view Jim, and I was remis in not making patches for 2.1.9 publicly announced and available[1], however, if you don't trust my 2.1.10 beta or rc release to be stable enough for production use, why would you think my patches for 2.1.9 would be any better?
That's a very interesting, and good, question. From my point of view, which may be different from others, it depends on the situation and need for patching. For instance, there is more than "./configure" and "make install" involved in a complex setup. Additionally, I have local patches for things that my sites need. Setting all that up takes time, and (in a normal day) if there is soon to be another release I have to pause and judge whether my time is best spent on RC1 or the Final build. NOTE: I, like most reading this, would devote much greater attention if there was a appearance of urgent need to test specific fixes. For the most part 2.1.10 (to me) appeared to be some behind the scenes XSS fixes and nothing more. So, assuming Development had it under control (and by all accounts they did), why would I spend 1 weekend setting up and testing RC1 when the Final would be out in 2 weeks and I would have to do all that effort again? Now if 2.1.10 was a code fix release for dying processes, and if my Mailman systems were experiencing dying processes, then my desire to test early and often would be driven by my desire to have a stable install (even at the RC level). However Mailman 2.1.9 has been very stable for me (THANK YOU) and so I don't know that I have anything to test in the RC that I won't be testing for in the Final. Hidden in that text is the admission that I trust you (Mark, Barry, etc.) to release 2.1.10 with as few of changes from 2.1.9 as necessary. If 2.1.10 were a complete re-write, then obviously my thoughts on this would be different. For the record, Mark, I would always be willing to at least look at future patches and give you a reasoned response as to whether I could even test it or not.
I really am faced with only two choices. Commit my fixes to the publicly available source tree so they can be exposed and tested in a wide variety of environments during the beta release phase, which process necessarily also exposes the vulnerabilities that they fix to the world, or sit on my patches and release them untested by others in the final release.
I can appreciate the significance of that situation. I don't know that I have a solution other than to ask what does ClamAV or SpamAssassin do in similar situations? I believe I shepherded the idea, some time ago, of the need for a closed Mailman security team of both developers and involved site administrators. I would say if a proven trusted group of Mailman site administrators privately discussed and tested a security fix, then I would have no problem with fixes being committed and released at once. Although a "heads up!" would be nice too. ;-)
[1]Patches for CVE-2008-0564 were made available to those who asked, and a google search will show that some distros have been patched, although Ubuntu for example <https://bugs.launchpad.net/ubuntu/+source/mailman/+bug/199338> calls it "low" importance.
Well, I gave up running Ubuntu on servers (although I still do on my laptop) specifically because I didn't like there approach to things like having NetworkManager installed/enabled by default on a Server install. ;-)
-Jim P.
On 4/16/08, Jim Popovitch wrote:
I can appreciate the significance of that situation. I don't know that I have a solution other than to ask what does ClamAV or SpamAssassin do in similar situations?
Dunno. Do they have to support twenty different languages?
Can those translations only be started once the mainline English-language code has been frozen, because the strings in the English-language version are used as the keys for the translated versions of the strings in other the other languages?
I believe I shepherded theidea, some time ago, of the need for a closed Mailman security team of both developers and involved site administrators.
There is one. See FAQ 1.27. It's secure enough that even I don't have access to it, although I'm sure that Barry, Mark, Tokio, and other critical people do.
I would say if aproven trusted group of Mailman site administrators privately discussed and tested a security fix, then I would have no problem with fixes being committed and released at once.
That's what you've got with the way RCs are handled today.
Although a "heads up!"would be nice too. ;-)
That's what you've got with the way RCs are handled today.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 16, 2008, at 12:21 AM, Jim Popovitch wrote:
I really am faced with only two choices. Commit my fixes to the publicly available source tree so they can be exposed and tested in a wide variety of environments during the beta release phase, which process necessarily also exposes the vulnerabilities that they fix to the world, or sit on my patches and release them untested by others
in the final release.I can appreciate the significance of that situation. I don't know that I have a solution other than to ask what does ClamAV or SpamAssassin do in similar situations? I believe I shepherded the idea, some time ago, of the need for a closed Mailman security team of both developers and involved site administrators. I would say if a proven trusted group of Mailman site administrators privately discussed and tested a security fix, then I would have no problem with fixes being committed and released at once. Although a "heads up!" would be nice too. ;-)
We have such a closed list, currently consisting of Mark, Tokio and
myself. It's who you get when you contact mailman-
security@python.org. More volunteers would probably be welcome,
especially if they were devoted to lending the additional help you
describe above. Note too that we don't work in a vacuum. Very often
we're working with vendor-sec to address security issues in a
responsible and coordinated way.
[1]Patches for CVE-2008-0564 were made available to those who asked, and a google search will show that some distros have been patched, although Ubuntu for example <https://bugs.launchpad.net/ubuntu/+source/mailman/+bug/199338> calls it "low" importance.
Well, I gave up running Ubuntu on servers (although I still do on my laptop) specifically because I didn't like there approach to things like having NetworkManager installed/enabled by default on a Server install. ;-)
BTW, it's not our responsibility to do anything other than patch the
Mailman source distribution. We'll work with vendors of course, but
it's really up to them to decide which patches to incorporate and and
how to distribute. If you don't want to run from source, you have to
trust your distro vendor to do the right thing.
Fortunately now, you have another option. You could track changes to
the master Bazaar repositories using your own branches. Then you can
decide which of our changes to cherry pick into your own running
servers, and easily merge in your own customization. Nobody's doing
it this way yet afaik, but I think it would work quite well for some
sites.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkgGiu8ACgkQ2YZpQepbvXGSUQCeIHdAwKEnUvVJc69B97/2gNgp GVwAn3bqBbCiXYZ0JxgRkvfUZNUSSvrQ =7rg6 -----END PGP SIGNATURE-----
Barry Warsaw writes:
BTW, it's not our responsibility to do anything other than patch the
Mailman source distribution.
I think you've missed at least part of Jim's point ...
Then you can decide which of our changes to cherry pick into your own running servers, and easily merge in your own customization.
Ayup, I do think you did. Over his boss's dead body he will ....
The two points he wants, I think, are
(1) the certification that comes with an Official Release, and
(2) Minimal Change (addressing *only* the security issues) from the current Official Stable Release. Maybe even a patch for the previous O.S.R., since many people give a release a bit of time to shake down.
*How* those changes get into his installation are (at this point) a secondary concern.
Jim?
On Thu, Apr 17, 2008 at 12:07 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Barry Warsaw writes:
BTW, it's not our responsibility to do anything other than patch the Mailman source distribution.
I think you've missed at least part of Jim's point ...
Then you can decide which of our changes to cherry pick into your own running servers, and easily merge in your own customization.
Ayup, I do think you did. Over his boss's dead body he will ....
The two points he wants, I think, are
(1) the certification that comes with an Official Release, and
(2) Minimal Change (addressing *only* the security issues) from the current Official Stable Release. Maybe even a patch for the previous O.S.R., since many people give a release a bit of time to shake down.
*How* those changes get into his installation are (at this point) a secondary concern.
Jim?
Correct. Security fixes should be minimal and quick, needing very little effort/attention by end users (i.e. Mailman operators). I would be very trusting and very happy if things like XSS and remote exploits were handled outside of CVS/SVN, then tested by a core group of operators to make sure the fixes didn't break other things. And then (same day) commits to CVS/SVN and source releases to the market. 2.1.10.rc1 appears to be more than security fixes, and as such is held up by language dependencies and other standard release issues. I think the process needs to change and have security issues handled outside of normal releases.
And for the record, I would be very willing to help out (i have python skils), but $DAYJOB legally prevents me from pretty much actively getting involved. Further, if I did contribute code, it could open Mailman up to legal issues. But, testing, etc, are ok because they are not IP related.
-Jim P.
On 4/17/08, Jim Popovitch wrote:
I think the process needs to change and have security issues handled outside of normal releases.
Which is what normally happens in the process as it currently exists. It's just that, in this particular case, this bug wasn't exposed until an earlier 2.1.10b version was released, and then we fixed this security hole.
So, in this case, to get the security fix you need to install the latest 2.1.10rc (which includes additional functionality), as opposed to a patch to a previous 2.1.9 version (which would presumably include just the security fix).
To go down the road you suggest would mean that we'd be responsible for back-porting all security-only fixes to all previous versions of Mailman, as a completely separate release tree from the new development work.
Speaking only for myself, this seems to be a significant additional amount of work, and I think it's unlikely to happen unless we get a lot more resources on this project. We'd need developers working on new code, developers working exclusively on security fixes, and a separate Release Engineer whose sole responsibility is to manage the process of creating appropriate patch releases as well as sheparding the new development releases.
FreeBSD can get away with that, because they've got a lot more people working on the project, and a lot more money supporting those people. I doubt we're ever going to be in a position to do something like that ourselves. In this project, most people have to wear multiple hats, and work on new development, security fixes, and release engineering, all at the same time.
And for the record, I would be very willing to help out (i have python skils), but $DAYJOB legally prevents me from pretty much actively getting involved. Further, if I did contribute code, it could open Mailman up to legal issues. But, testing, etc, are ok because they are not IP related.
You could take over the Release Engineering job, and manage the two separate security patch-only releases as well as the new-development releases.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 15, 2008, at 9:18 PM, Jim Popovitch wrote:
On Tue, Apr 15, 2008 at 9:10 PM, Barry Warsaw <barry@python.org>
wrote:Better to get a release out asap after that and let the community
know that there are important fixes contained within.Fair enough. Where's the release then?
Look, I know you folks are working hard on this, and I certainly don't dis-respect that. HOWEVER, the process flow needs some re-thinking. You should not publicly release security vulnerability details before fixes are identified for current versions. I can't imagine that you don't already know that.
There is some validity to the complaint that new releases are blocked
on translation updates. Our translators do a wonderful, and greatly
appreciated job, but they're disadvantaged by our suboptimal
translation process. A big motivation for moving to a service like
Pootle or Launchpad translations is to help decouple their work from
that of the release manager. I think we also need to find a better
way to package non-English language support so that updates to
languages can happen on their own schedule without requiring or
blocking the core Mailman releases.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkgGiDsACgkQ2YZpQepbvXEk/wCfVPpJvVHt+C8rSt1Tpsgsx4oe SBcAni/CWLPC8yhTNRx/wjUVyePlfzNQ =pQnR -----END PGP SIGNATURE-----
Barry Warsaw writes:
There is some validity to the complaint that new releases are blocked
on translation updates. Our translators do a wonderful, and greatly
appreciated job, but they're disadvantaged by our suboptimal
translation process.
Fixing that won't help security releases, though. I don't recall anybody waiting on translations for the Apache traversal bug.
I think we also need to find a better way to package non-English language support so that updates to languages can happen on their own schedule without requiring or blocking the core Mailman releases.
How do you propose to do that, except by releasing English-only versions first?
I don't know the details of Python's implementation, but the gettext C library is already pretty close to optimal on this, isn't it? You just drop a random .mo file into the right place, and gettext will use the translation it finds if there's one there, and otherwise spew the original string.
So all you really need is a utility to uninstall (and optionally back up) the old language file and install the new one in the right place.
BTW, the *real* problem here is that we *really* need to free up Mark for doing more development. Whether he likes it or not. ;-)
Jim Popovitch writes:
On Tue, Apr 15, 2008 at 11:56 AM, Mark Sapiro <mark@msapiro.net> wrote:
There are two security issues mentioned in the announcement.
<harsh criticism> How much sense does it make to announce security issues in a release CANDIDATE? Come on guys, release a STABLE version (or FIX), then announce. <--- Standard Operation Procedures. </harsh criticism>
Quit feeding the enemy instead of your supporters.
That last was *quite* uncalled-for.
First of all, there is no such thing as a STABLE security release (you can't beta test it before announcing and you don't have time to alpha test properly), so it can't possibly be S.O.P.
Second, the fact that you emphasize STABLE indicates (and you confirm) that you're thinking of a situation where you can't upgrade to an unstable version because of "rules". That's not Mark's problem, that's *your* problem if you are not trusted to judge whether a fix is sufficiently important and stable to be applied. I understand your frustration, but it ain't Mark's fault.
Third, you have real problems anyway. Mailman has known security bugs that are not going to get fixed soon (specifically the LIST-request backscatter problem).[1] You are aware of that, right? In the great scheme of things, I don't think the two minor issues fixed in this RC are a really big deal....
Anyway, whether you like it or not, the backscatter issue shows that the Mailman project currently does not put #1 priority on things just because they're labeled "security issues". If you want that changed, it would help to have a better idea of what exactly is most important, because resources aren't infinite.
Now, your desire for a stable security release suggests that what you want is a procedure like
(0) mailman-security@python.org receives a report. (1) The security cabal prepares a fix to the current stable release (if necessary, cutting a new branch) and tests it as well as possible. (2) The release is cut as version STABLE.SECURITY-PATCH-NUMBER. (3) The release is announced, and (optionally) a patch published for the DIY crowd. (4) The developers consider whether to forward port or design a different fix for in-development versions (including the next stable/bugfix release).
Is that right? If so, how much are you (and others) willing to contribute toward achieving that process? Eg, maybe you could beat Mark to the punch on a few of the routine questions that he must spend 5 hours a week on, or so? Or if you have Python skills, you could volunteer to burn some midnight oil on generating patches and alpha testing? Etc, etc.
Footnotes: [1] True, with some effort you can shut those aliases off, but that will invalidate many of the information web pages, and for that reason the secure configuration has not been made default, and probably that will be postponed to Mailman 2.2.
Stephen J. Turnbull wrote:
[1] True, with some effort you can shut those aliases off, but that will invalidate many of the information web pages, and for that reason the secure configuration has not been made default, and probably that will be postponed to Mailman 2.2.
I'd be really surprised if that ever becomes the default, because there are so many other ways in which backscatter can be generated -- either by Mailman, or any other MLM.
Yes, in the future I would hope there would be options available to the site admin, or ideally also to the list admin, to reduce backscatter generation, but they should address all potential sources of backscatter and not just the aliases.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I am happy to announce the release of Mailman 2.1.10.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support for three new language translations, Galician, Hebrew and Slovak and a few new features.
Mailman is free software for managing email mailing lists and e- newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites.
For more information, including download links, please see:
http://www.list.org http://mailman.sf.net http://www.gnu.org/software/mailman
Special thanks are due to Barry Warsaw and Tokio Kikuchi for much coding and support, Moritz Naumann for help with security issues and Jim Tittsler for a significant patch.
Here's a list of the major changes.
Security
- The 2.1.9 fixes for CVE-2006-3636 were not complete. In particular, some potential cross-site scripting attacks were not detected in editing templates and updating the list's info attribute via the web admin interface. This has been assigned CVE-2008-0564 and has been fixed. Thanks again to Moritz Naumann for assistance with this.
- There is a new mm_cfg.py/Defaults.py variable OWNERS_CAN_CHANGE_MEMBER_PASSWORDS which controls whether the list owner can change a member's password from the member's options page. This defaults to No and should be changed to Yes only if list owners are trusted to not change a member's password, log in as the member and make global membership changes.
Note: If you are not ready to upgrade, patches for these two issues are available at http://sourceforge.net/project/showfiles.php?group_id=103 in the 2.1.9 file list.
New Features
- Changed cmd_who.py to list all members if authorization is with the list's admin or moderator password and to accept the password if the roster is public. Also changed the web roster to show hidden members when authorization is by site or list's admin or moderator password (1587651).
- Added the ability to put a list name in accept_these_nonmembers to accept posts from members of that list (1220144).
- Added a new 'sibling list' feature to exclude members of another list from receiving a post from this list if the other list is in the To: or Cc: of the post or to include members of the other list if that list is not in the To: or Cc: of the post (Patch ID 1347962).
- Added the admin_member_chunksize attribute to the admin General Options interface (Bug 1072002, Partial RFE 782436).
Internationalization
- Added the Hebrew translation from Dov Zamir. This includes addition of a direction ('ltr', 'rtl') to the LC_DESCRIPTIONS table. The add_language() function defaults direction to 'ltr' to not break existing mm_cfg.py files.
- Added the Slovak translation from Martin Matuska.
- Added the Galician translation from Frco. Javier Rial RodrÃguez.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFIDO/oVVuXXpU7hpMRAngiAKCwIOhSJJrCaY3afhGJQN339/dKuACeMckR SI7DwCHcYONnIj3LoNYueC0= =DO/d -----END PGP SIGNATURE-----
On Mon, Apr 21, 2008 at 3:50 PM, Mark Sapiro <mark@msapiro.net> wrote:
Note: If you are not ready to upgrade, patches for these two issues are available at http://sourceforge.net/project/showfiles.php?group_id=103 in the 2.1.9 file list.
THANK YOU!
-Jim P.
Mark Sapiro wrote:
I am happy to announce the release of Mailman 2.1.10.
Now running on mail.python.org. Please let us know if there are any problems.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 21, 2008, at 5:28 PM, Brad Knowles wrote:
Mark Sapiro wrote:
I am happy to announce the release of Mailman 2.1.10.
Now running on mail.python.org. Please let us know if there are any
problems.
Wow, awesome Brad. Thanks!
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkgNFi4ACgkQ2YZpQepbvXG4+wCgp+P2SypS4dBTbAdP4LBpShDu VCoAoKn0Xvqlt0x72zOWgsvlQA1cJLMv =4M3O -----END PGP SIGNATURE-----
Mark Sapiro wrote:
I am happy to announce the release of Mailman 2.1.10.
Congratulations!
I presume the usual upgrade instructions apply?
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.014.htp
dn
David Newman wrote:
I presume the usual upgrade instructions apply?
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.014.htp
For upgrading python.org, I made the mistake of leaving Mailman running even though I had shut down postfix and apache, and it still worked okay. I'd recommend shutting down the web interface first, then Mailman, and then finally the MTA.
Then do the upgrade, and restart them in reverse order.
Then watch your log files like a hawk, and test out all your standard "stupid user tricks" to see if anything obvious has been broken.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
David Newman wrote:
I presume the usual upgrade instructions apply?
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.014.htp
Those are really somewhat outdated. I probably should have just replaced them, but I created a new FAQ <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.080.htp>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
I am happy to announce the release of Mailman 2.1.10.
This is a security and bug fix release and it is highly recommended that all sites upgrade to this version. Mailman 2.1.10 also adds support for three new language translations, Galician, Hebrew and Slovak and a few new features.
Remind me next time that we *MUST* upgrade to the betas and RCs on python.org once you've made them available. The changes made to the code which supports skipping unparseable messages means that mmdsr has to be changed to suit, otherwise you could wind up with a daily report that is 800MB in size, depending on how many "unparseable messages" you might have.
We'll need to coordinate the updates to mmdsr and get the official version with all code contributions from all parties out there on the SourceForge page.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
Brad Knowles wrote:
Remind me next time that we *MUST* upgrade to the betas and RCs on python.org once you've made them available. The changes made to the code which supports skipping unparseable messages means that mmdsr has to be changed to suit, otherwise you could wind up with a daily report that is 800MB in size, depending on how many "unparseable messages" you might have.
We'll need to coordinate the updates to mmdsr and get the official version with all code contributions from all parties out there on the SourceForge page.
OK, I'll try to remember.
FYI, I've been running all the 2.1.10 beta and rc releases and mmdsr and haven't seen an issue, but I think in my case Postgrey/MailScanner must be getting rid of all the unparseable messages before they reach Mailman.
Note that in addition to the logging change for unparseable messages, they will now be saved in the shunt queue, so if you have a lot, you'll have to deal with that too.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Note that in addition to the logging change for unparseable messages, they will now be saved in the shunt queue, so if you have a lot, you'll have to deal with that too.
These are the *.psv files? Yeah, we've got almost 8000 of them on python.org. Do you have any documentation on what should be done with them?
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
Brad Knowles wrote:
Mark Sapiro wrote:
Note that in addition to the logging change for unparseable messages, they will now be saved in the shunt queue, so if you have a lot, you'll have to deal with that too.
These are the *.psv files? Yeah, we've got almost 8000 of them on python.org. Do you have any documentation on what should be done with them?
Wow! I never thought there would be anything like this.
Here's the situation. Pre 2.1.9, when Mailman dequeued an incoming message and then the Python email library couldn't parse the MIME structure the message was just lost. There wasn't anything that could be done other than log the fact. This is not a big deal as the message's MIME structure was defective and the message was almost certainly spam.
Beginning in 2.1.9, we implemented the .bak files to back up an in process message so the message could be recovered if something died horribly (e.g. power failure) while it was in process. Nothing was done at that time about the unparseable messages.
I then realized that the unparseable message had an intact .bak file in the queue, so for 2.1.10, I decided to preserve this file (thus the .psv in the shunt queue) in case some human wanted to look at it with bin/dumpdb or whatever. Obviously, if you get 8000+ in a few hours, no one is going to look at them all or even much of a sample.
You could set up a cron to run every hour or some other interval to efectively do
rm $var_prefix/qfiles/shunt/*.psv
The problem with that is there can occasionally be queue entries preserved for other conditions which are hopefully much rarer, but you might actually want to look at those.
I think the best solution is to turn off the preservation of unparseable messages, and add an mm_cfg.py setting to turn it on. I can work up a patch.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark Sapiro wrote: | | I think the best solution is to turn off the preservation of | unparseable messages, and add an mm_cfg.py setting to turn it on. I | can work up a patch. | A patch is attached. It doesn't turn off preservation by default because it piggybacks on an existing Defaults.py/mm_cfg.py setting which was lately used only by bin/update. The Defaults.py setting is QRUNNER_SAVE_BAD_MESSAGES = Yes Setting QRUNNER_SAVE_BAD_MESSAGES = No in mm_cfg.py will stop preserving the unparseable messages, but certain other problem qfiles will still be preserved. Additionally, preserved files will be stored in qfiles/bad and not qfiles/shunt. Finally, if QRUNNER_SAVE_BAD_MESSAGES = No, the log message will revert to the old "Ignoring unparseable message:". Brad, If you could apply this patch (you can apply it directly to the installation directory, by e.g. cd /usr/local/mailman patch -p0 < path/to/2.1.10.patch.txt and set QRUNNER_SAVE_BAD_MESSAGES = No in mm_cfg.py and restart Mailman, things should be back more or less the way they were in 2.1.9. I have applied the patch to my installation and I'm sure it's good, but I haven't seen any unparseable messages. - -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFIDVAhVVuXXpU7hpMRAqHBAKCFODqF84UhBYnRvblS00eX4roxdQCeNOXl amZX+XRwrdd9RpIE80eOFKM= =vi+P -----END PGP SIGNATURE----- === modified file 'Mailman/Queue/Runner.py' --- Mailman/Queue/Runner.py 2007-05-08 03:16:04 +0000 +++ Mailman/Queue/Runner.py 2008-04-22 02:02:38 +0000 @@ -1,4 +1,4 @@ -# Copyright (C) 1998-2007 by the Free Software Foundation, Inc. +# Copyright (C) 1998-2008 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License @@ -103,12 +103,18 @@ # but other problems can occur in message parsing, e.g. # ValueError, and exceptions can occur in unpickling too. # We don't want the runner to die, so we just log and skip - # this entry, but preserve it for analysis. + # this entry, but maybe preserve it for analysis. self._log(e) - syslog('error', - 'Skipping and preserving unparseable message: %s', - filebase) - self._switchboard.finish(filebase, preserve=True) + if mm_cfg.QRUNNER_SAVE_BAD_MESSAGES: + syslog('error', + 'Skipping and preserving unparseable message: %s', + filebase) + preserve=True + else: + syslog('error', + 'Ignoring unparseable message: %s', filebase) + preserve=False + self._switchboard.finish(filebase, preserve=preserve) continue try: self._onefile(msg, msgdata) === modified file 'Mailman/Queue/Switchboard.py' --- Mailman/Queue/Switchboard.py 2008-02-14 16:53:52 +0000 +++ Mailman/Queue/Switchboard.py 2008-04-22 02:11:19 +0000 @@ -169,13 +169,13 @@ bakfile = os.path.join(self.__whichq, filebase + '.bak') try: if preserve: - psvfile = os.path.join(mm_cfg.SHUNTQUEUE_DIR, filebase + '.psv') + psvfile = os.path.join(mm_cfg.BADQUEUE_DIR, filebase + '.psv') # Create the directory if it doesn't yet exist. # Copied from __init__. omask = os.umask(0) # rwxrws--- try: try: - os.mkdir(mm_cfg.SHUNTQUEUE_DIR, 0770) + os.mkdir(mm_cfg.BADQUEUE_DIR, 0770) except OSError, e: if e.errno <> errno.EEXIST: raise finally:
On 4/21/08, Mark Sapiro wrote:
If you could apply this patch (you can apply it directly to the installation directory, by e.g.
cd /usr/local/mailman patch -p0 < path/to/2.1.10.patch.txt
Patch applied fine, no complaints.
and set
QRUNNER_SAVE_BAD_MESSAGES = No
Done.
in mm_cfg.py and restart Mailman, things should be back more or less the way they were in 2.1.9.
I have applied the patch to my installation and I'm sure it's good, but I haven't seen any unparseable messages.
I haven't seen any more unparseable messages in the last few minutes, but let's see how things go.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
On 4/21/08, Brad Knowles wrote:
I have applied the patch to my installation and I'm sure it's good, but I haven't seen any unparseable messages.
I haven't seen any more unparseable messages in the last few minutes, but let's see how things go.
We've now had a couple of unparseable messages since applying the patch, and it seems to be working as expected.
Going back to Jun 12 23:48:51 2007, it looks like we've had a total of about 378,989 unparseable messages, although we only have just over 8000 messages in the mailman/qfiles/shunt and mailman/qfiles/shunt.old directories (7759 .psv files, and 312 .pck files).
That works out to about 37,900 unparseable messages per month, or something like about 125 unparseable messages per day.
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
Brad Knowles wrote:
On 4/21/08, Brad Knowles wrote:
I have applied the patch to my installation and I'm sure it's good, but I haven't seen any unparseable messages.
I haven't seen any more unparseable messages in the last few minutes, but let's see how things go.
We've now had a couple of unparseable messages since applying the patch, and it seems to be working as expected.
Going back to Jun 12 23:48:51 2007, it looks like we've had a total of about 378,989 unparseable messages, although we only have just over 8000 messages in the mailman/qfiles/shunt and mailman/qfiles/shunt.old directories (7759 .psv files, and 312 .pck files).
That works out to about 37,900 unparseable messages per month, or something like about 125 unparseable messages per day.
The light dawns! (thanks to the post at <http://mail.python.org/pipermail/mailman-users/2008-April/061365.html>).
There was an error in 2.1.9 that didn't delete (or move aside) the .bak file in qfiles/in from an unparseable message. Thus, there were 7500+ unparseable message qfiles/in/*.bak files when you restarted mailman. These were all 'recovered' and reprocessed, reproducing the unparseable message error, and this time moved to qfiles/shunt/*.psv (so it wouldn't happen again).
This explains why there were over 7500 of these at once, when the normal rate is /only/ 125 per day.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Brad Knowles wrote:
Going back to Jun 12 23:48:51 2007, it looks like we've had a total of about 378,989 unparseable messages, although we only have just over 8000 messages in the mailman/qfiles/shunt and mailman/qfiles/shunt.old directories (7759 .psv files, and 312 .pck files).
There are two reasons why you've had 378,989 unparseable messages, but only have 7759 .psv files.
First, some of the unparseable .bak files that 2.1.9 left behind in qfiles/in may have been removed or moved aside.
Second, whenever Mailman was restarted, any qfiles/in/*.bak files would have been reprocessed resulting in a flood of unparseable message errors and leaving just one new .bak file per message. Thus a single message and .bak file could have been responsible for several logged unparseable message errors if Mailman had been restarted several times.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 21, 2008, at 9:30 PM, Mark Sapiro wrote:
You could set up a cron to run every hour or some other interval to efectively do
rm $var_prefix/qfiles/shunt/*.psv
The problem with that is there can occasionally be queue entries preserved for other conditions which are hopefully much rarer, but you might actually want to look at those.
I think the best solution is to turn off the preservation of unparseable messages, and add an mm_cfg.py setting to turn it on. I can work up a patch.
We should probably have some kind of shunt queue culler cron script in
place, either that archives and deletes those files, or just expires
them after a certain amount of time. What to people generally do with
their shunt files?
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkgNUlsACgkQ2YZpQepbvXFI8gCgpF+Z1ROdfrEQa4ACrUnDBkJT DWIAn0qyRtYt4/1UCCpKSmImyLAWhkZO =WmVk -----END PGP SIGNATURE-----
On 4/21/08, Barry Warsaw wrote:
We should probably have some kind of shunt queue culler cron script in place, either that archives and deletes those files, or just expires them after a certain amount of time.
That's easy enough to do with cron and find. You tell me what you want, and I'll be glad to set that up.
What to people generally do withtheir shunt files?
Leave them untouched for months or years? ;-)
-- Brad Knowles <brad@shub-internet.org> LinkedIn Profile: <http://tinyurl.com/y8kpxu>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Apr 21, 2008, at 3:50 PM, Mark Sapiro wrote:
I am happy to announce the release of Mailman 2.1.10.
Congratulations Mark! Long live Mailman 2.2. :)
I will update the web sites.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkgNFc0ACgkQ2YZpQepbvXEQ0wCePrsNZ1cyXStsBpjMHR94o20H HoEAn3Fv8D3WC3NCSkkjg9qIS5I5CzzP =1UG9 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark Sapiro wrote: | I am happy to announce the release of Mailman 2.1.10. I have discovered a few problems with the release. None is a major show stopper, but the most significant so far is that I broke cmd_subscribe so that email subscribe to the -subscribe or -join address or the - -request address with a bare 'subscribe' command results in the message being shunted. A patch for this is attached, but I plan to make a patch release probably next week. - -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFID1HvVVuXXpU7hpMRAm+3AKD2otRNTXWYSRjguJEYVc0HRVflhACZAVXf Kao5NWvpiRQ9U9keKT2Jbj8= =1NSl -----END PGP SIGNATURE----- === modified file 'Mailman/Commands/cmd_subscribe.py' --- Mailman/Commands/cmd_subscribe.py 2008-03-20 03:07:51 +0000 +++ Mailman/Commands/cmd_subscribe.py 2008-04-23 14:32:48 +0000 @@ -71,7 +71,8 @@ return STOP argnum += 1 # Fix the password/digest issue - if digest is None and password.lower() in ('digest', 'nodigest'): + if (digest is None + and password and password.lower() in ('digest', 'nodigest')): if password.lower() == 'digest': digest = 1 else:
participants (9)
-
Barry Warsaw -
Barry Warsaw -
Brad Knowles -
Christopher X. Candreva -
David Newman -
Dragon -
Jim Popovitch -
Mark Sapiro -
Stephen J. Turnbull