Mailman 3 October 2004 - Mailman-coders

[ mailman-Bugs-1054944 ] Default templates set bgcolors
by SourceForge.net 26 Oct '04

26 Oct '04

Bugs item #1054944, was opened at 2004-10-26 23:58 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1054944&group_… Category: Web/CGI Group: 2.1 (stable) Status: Open Resolution: None Priority: 5 Submitted By: MJ Ray (slef) Assigned to: Nobody/Anonymous (nobody) Summary: Default templates set bgcolors Initial Comment: Many of the templates for the CGI and for the archives set background colours but do not set text colours. This leads to readers who don't use dark-on-light default colours getting very hard-to-read pages. Mailman is not following Web Content Accessibility Guideline 2.2 I attach a patch which fixes templates/en in a minimal way. I hope that's OK. If not, please direct me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1054944&group_…

1 0

[ mailman-Patches-1045656 ] Race in LockFile.py resulting in extraneous log entries.
by SourceForge.net 26 Oct '04

26 Oct '04

Patches item #1045656, was opened at 2004-10-12 14:50 Message generated for change (Comment added) made by grnbrg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_… Category: mail delivery Group: Mailman 2.1 Status: Open Resolution: Postponed Priority: 3 Submitted By: Brian Greenberg (grnbrg) Assigned to: Tokio Kikuchi (tkikuchi) Summary: Race in LockFile.py resulting in extraneous log entries. Initial Comment: There is a race condition (that does not affect correct operation of Mailman) in .../Mailman/LockFile.py. This race results in entries to the "locks" log file (usually in pairs) that are unnecessary and confusing. If there is a process holding a lock for a file, and a process waiting for that lock to be freed, the following sequence can occur: 1) The waiting process executes os.link(). Since the running process still has the lock, this fails with EEXIST. 2) The running process releases the lock, and removes the lock file. 3) The waiting process proceeds with it's checks, and checks self.__linkcount() which returns -1 due to an ENOENT error. This throws an error into the "locks" log file. 4) The waiting process continues processing, checks it's timeout, and then checks to see if the lock lifetime has expired. The lifetime check is based on a call to self.__releasetime(), which returns -1, due to an ENOENT error. This results in the lockfile being declared stale, and self.__break() is called. This then throws a second error into the "locks" log file. The attached patch checks for these ENOENT errors, immediately restarting the loop when they are detected. It does not affect operation is the lock file exists and is held by another process. Brian Greenberg. ---------------------------------------------------------------------- >Comment By: Brian Greenberg (grnbrg) Date: 2004-10-26 11:46 Message: Logged In: YES user_id=902583 I'm running Solaris 8, with the locks and qfiles on local disk. mm 2.1.5 and python 2.3.3. There are a little over 1000 lists, with traffic of 300-500 posts per day, on average. Given that it's cosmetic, there is no rush. I'm open to testing suggestions... ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-24 19:02 Message: Logged In: YES user_id=67709 OK. I found the log output in my solaris installation. But, I can't reproduce the said log-pair using bin/withlist. It looks like occurrence of this log output is unpredictable and need more checking before applying the patch. I found [unexpected linkcount: -1/lifetime has expired, breaking] pair in Solaris [lifetime has expired, breaking] only in BSD/OS no locks log in FreeBSD So, the behaviour is highly OS dependent I guess. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2004-10-24 03:02 Message: Logged In: YES user_id=75166 The comments in the LockFile.py header code make the point that the entire mechanism is to deal with NFS robustness/compatibilty issues. The extranenous logging that concerns grnbrg is appearing on Solaris (version and build unspecified) and attempts to reproduce it, that have thus far failed, appear to be on FreeBSD. I suspect that the majority of MM installs are on various versions and builds of Linux. These OSen will all have substantially different NFS implementations and major differences in their kernel that probably affect NFS implementation and operation. Because of these differences I have already encountered sufficient problems with interworking of Linux NFS clients with Solaris NFS servers to convince me that changing LockFile.py without sufficient testing in order to clear an essentially cosmetic problem on one host OS is a an unecessary risk to stability across the board. Just testing on one host is not good enough and reading code even less so; it is the dynamics of operation in live situations that are the issue. A test programme needs to at least nod in the direction of heterogenous NFS client/host configurations of various types being in used for validation and verification of any changes. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-22 13:53 Message: Logged In: YES user_id=902583 I'm running on a Solaris box, which may or may not have any impact on the issue. I'm not sure how to force reproduction -- races are hard to intentially duplicate. I first noticed the problem under a heavy load due to an admin "oops" -- 30,000+ news articles being gatewayed into a test list. :) Once the oops was fixed, the error logs continued to occasionally appear, and rather than assume that everything was ok, I tracked down the source of the problem. I did this by desk-checking the source, and identified the race by a bit of guesswork. Once these changes were made, the problem stopped. The race occurs as follows: 1) A qrunner runs LockFile.lock() to request a particular lock. 2) At LockFile.py:256, this qrunner attempts to link it's temp lock file to the global lockfile. This link fails with a EEXIST error, because another process currently holds the lock. The assumption is made at this point that the global lockfile exists. This (false) assumption continues through to the end of the loop at LockFile.py:312. In fact, the global lockfile may cease to exist (because the qrunner that has it calls unlock()) at any time. 3) The process holding the lock calls unlock(). 4) The waiting process (which just got an EEXIST) goes through it's various sanity checks, which include checking the linkcount and mtime of the global lockfile, which now does not exist. Since it is assumed to exist, the -1 results returned make no sense, and are unconditionally logged. 5) At LockFile.py:303, the (incorrectly assumed) existing lock is broken, and control goes back to the top of the loop (after a quick __sleep()) As far as impact, the patch should have minimal side effects. I pulled the function calls out of the if statements, so there is no change in the number of calls. Further, all the new code does is to check for the "-1" return that indicates that the global lockfile no longer exists, and immediately restarts the loop if that is the case. It might not be a bad idea to add a "self.__sleep()" before the continue, but I didn't think it was needed. Hope that adds enough detail to trace what I think is happening, and verify that it won't break anything. FWIW, I have not had any problems, nor any errors logged since applying this locally. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2004-10-22 07:06 Message: Logged In: YES user_id=12800 I haven't had time to look at this in detail, but I'll just include a general warning that LockFile.py is /very/ fragile. It's also critical to the proper operation of Mailman. So any changes, even seemingly innocent onces, must be very thoroughly tested. My inclination would be to reject any change that wasn't absolutely necessary, but I won't veto this change if it is well-tested and fixes a verified bug (tho' it sounds like tkikuchi can't reproduce it). ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 03:26 Message: Logged In: YES user_id=67709 Will you be more specific how this race occurs? My installation on FreeBSD doesn't looks like show this symptom in locks log file. I tested with two terminals invoking python -i bin/withlist testlist and try to lock with m.Lock() in both the terminal. One of term is forced to wait until the I type m.Unlock() in the other term. I then looked into logs dir but nothing avail. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-15 09:22 Message: Logged In: YES user_id=902583 This would fix this bug, as long as CLOCK_SLOP is large enough. But I don't think it's the Right Thing, as you are returning a result that says that the lockfile has a lifetime, when it in fact simply no longer exists. For a function whose purpose is to report the mtime of the master lockfile, -1 is the correct response if the file doesn't exist. The code that calls that function should check for and interpret that condition. ---------------------------------------------------------------------- Comment By: SHIGENO Kazutaka (shigeno) Date: 2004-10-14 19:11 Message: Logged In: YES user_id=1138453 How about changing as follows about self.__releasetime()? @@ -448,7 +448,7 @@ return os.stat(self.__lockfile)[ST_MTIME] except OSError, e: if e.errno <> errno.ENOENT: raise - return -1 + return time.time() def __linkcount(self): try: ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_…

1 0

[ mailman-Patches-1053558 ] i18n: Header Filter Rules (& fix)
by SourceForge.net 26 Oct '04

26 Oct '04

Patches item #1053558, was opened at 2004-10-25 16:35 Message generated for change (Comment added) made by hatukanezumi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053558&group_… Category: internationalization Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Hatuka*nezumi (hatukanezumi) Assigned to: Nobody/Anonymous (nobody) Summary: i18n: Header Filter Rules (& fix) Initial Comment: - Decode headers to be matched. - Normalize header & pattern so that compatibility characters (fullwidth forms of ASCII, Compatibility Ideographs etc.) will be matched. Normalization Form KC (NFKC) is used. note: This feature is available on Python >= 2.3. - Fix: Ignore empty lines in pattern to prevent matching any strings. ---------------------------------------------------------------------- >Comment By: Hatuka*nezumi (hatukanezumi) Date: 2004-10-26 13:21 Message: Logged In: YES user_id=529503 Error handlings are added. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053558&group_…

1 0

[ mailman-Patches-1053739 ] Fix colors in HTML pages
by SourceForge.net 25 Oct '04

25 Oct '04

Patches item #1053739, was opened at 2004-10-25 15:57 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053739&group_… Category: Web UI Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Nico (nico18) Assigned to: Nobody/Anonymous (nobody) Summary: Fix colors in HTML pages Initial Comment: The HTML pages in mailman-2.1 set the background color, but not the foreground color. This results in pages which are not readable if you have set your default colors in your browser to white on black, for example. This patch against current mailman-2.1 CVS fixes that bug by setting the text color to black in pages in which the background color is set to white. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053739&group_…

1 0

[ mailman-Patches-1053558 ] i18n: Header Filter Rules (& fix)
by SourceForge.net 25 Oct '04

25 Oct '04

Patches item #1053558, was opened at 2004-10-25 16:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053558&group_… Category: internationalization Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Hatuka*nezumi (hatukanezumi) Assigned to: Nobody/Anonymous (nobody) Summary: i18n: Header Filter Rules (& fix) Initial Comment: - Decode headers to be matched. - Normalize header & pattern so that compatibility characters (fullwidth forms of ASCII, Compatibility Ideographs etc.) will be matched. Normalization Form KC (NFKC) is used. note: This feature is available on Python >= 2.3. - Fix: Ignore empty lines in pattern to prevent matching any strings. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1053558&group_…

1 0

[ mailman-Patches-1045656 ] Race in LockFile.py resulting in extraneous log entries.
by SourceForge.net 25 Oct '04

25 Oct '04

Patches item #1045656, was opened at 2004-10-12 19:50 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_… Category: mail delivery Group: Mailman 2.1 Status: Open >Resolution: Postponed Priority: 3 Submitted By: Brian Greenberg (grnbrg) Assigned to: Tokio Kikuchi (tkikuchi) Summary: Race in LockFile.py resulting in extraneous log entries. Initial Comment: There is a race condition (that does not affect correct operation of Mailman) in .../Mailman/LockFile.py. This race results in entries to the "locks" log file (usually in pairs) that are unnecessary and confusing. If there is a process holding a lock for a file, and a process waiting for that lock to be freed, the following sequence can occur: 1) The waiting process executes os.link(). Since the running process still has the lock, this fails with EEXIST. 2) The running process releases the lock, and removes the lock file. 3) The waiting process proceeds with it's checks, and checks self.__linkcount() which returns -1 due to an ENOENT error. This throws an error into the "locks" log file. 4) The waiting process continues processing, checks it's timeout, and then checks to see if the lock lifetime has expired. The lifetime check is based on a call to self.__releasetime(), which returns -1, due to an ENOENT error. This results in the lockfile being declared stale, and self.__break() is called. This then throws a second error into the "locks" log file. The attached patch checks for these ENOENT errors, immediately restarting the loop when they are detected. It does not affect operation is the lock file exists and is held by another process. Brian Greenberg. ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-25 00:02 Message: Logged In: YES user_id=67709 OK. I found the log output in my solaris installation. But, I can't reproduce the said log-pair using bin/withlist. It looks like occurrence of this log output is unpredictable and need more checking before applying the patch. I found [unexpected linkcount: -1/lifetime has expired, breaking] pair in Solaris [lifetime has expired, breaking] only in BSD/OS no locks log in FreeBSD So, the behaviour is highly OS dependent I guess. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2004-10-24 08:02 Message: Logged In: YES user_id=75166 The comments in the LockFile.py header code make the point that the entire mechanism is to deal with NFS robustness/compatibilty issues. The extranenous logging that concerns grnbrg is appearing on Solaris (version and build unspecified) and attempts to reproduce it, that have thus far failed, appear to be on FreeBSD. I suspect that the majority of MM installs are on various versions and builds of Linux. These OSen will all have substantially different NFS implementations and major differences in their kernel that probably affect NFS implementation and operation. Because of these differences I have already encountered sufficient problems with interworking of Linux NFS clients with Solaris NFS servers to convince me that changing LockFile.py without sufficient testing in order to clear an essentially cosmetic problem on one host OS is a an unecessary risk to stability across the board. Just testing on one host is not good enough and reading code even less so; it is the dynamics of operation in live situations that are the issue. A test programme needs to at least nod in the direction of heterogenous NFS client/host configurations of various types being in used for validation and verification of any changes. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-22 18:53 Message: Logged In: YES user_id=902583 I'm running on a Solaris box, which may or may not have any impact on the issue. I'm not sure how to force reproduction -- races are hard to intentially duplicate. I first noticed the problem under a heavy load due to an admin "oops" -- 30,000+ news articles being gatewayed into a test list. :) Once the oops was fixed, the error logs continued to occasionally appear, and rather than assume that everything was ok, I tracked down the source of the problem. I did this by desk-checking the source, and identified the race by a bit of guesswork. Once these changes were made, the problem stopped. The race occurs as follows: 1) A qrunner runs LockFile.lock() to request a particular lock. 2) At LockFile.py:256, this qrunner attempts to link it's temp lock file to the global lockfile. This link fails with a EEXIST error, because another process currently holds the lock. The assumption is made at this point that the global lockfile exists. This (false) assumption continues through to the end of the loop at LockFile.py:312. In fact, the global lockfile may cease to exist (because the qrunner that has it calls unlock()) at any time. 3) The process holding the lock calls unlock(). 4) The waiting process (which just got an EEXIST) goes through it's various sanity checks, which include checking the linkcount and mtime of the global lockfile, which now does not exist. Since it is assumed to exist, the -1 results returned make no sense, and are unconditionally logged. 5) At LockFile.py:303, the (incorrectly assumed) existing lock is broken, and control goes back to the top of the loop (after a quick __sleep()) As far as impact, the patch should have minimal side effects. I pulled the function calls out of the if statements, so there is no change in the number of calls. Further, all the new code does is to check for the "-1" return that indicates that the global lockfile no longer exists, and immediately restarts the loop if that is the case. It might not be a bad idea to add a "self.__sleep()" before the continue, but I didn't think it was needed. Hope that adds enough detail to trace what I think is happening, and verify that it won't break anything. FWIW, I have not had any problems, nor any errors logged since applying this locally. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2004-10-22 12:06 Message: Logged In: YES user_id=12800 I haven't had time to look at this in detail, but I'll just include a general warning that LockFile.py is /very/ fragile. It's also critical to the proper operation of Mailman. So any changes, even seemingly innocent onces, must be very thoroughly tested. My inclination would be to reject any change that wasn't absolutely necessary, but I won't veto this change if it is well-tested and fixes a verified bug (tho' it sounds like tkikuchi can't reproduce it). ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 08:26 Message: Logged In: YES user_id=67709 Will you be more specific how this race occurs? My installation on FreeBSD doesn't looks like show this symptom in locks log file. I tested with two terminals invoking python -i bin/withlist testlist and try to lock with m.Lock() in both the terminal. One of term is forced to wait until the I type m.Unlock() in the other term. I then looked into logs dir but nothing avail. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-15 14:22 Message: Logged In: YES user_id=902583 This would fix this bug, as long as CLOCK_SLOP is large enough. But I don't think it's the Right Thing, as you are returning a result that says that the lockfile has a lifetime, when it in fact simply no longer exists. For a function whose purpose is to report the mtime of the master lockfile, -1 is the correct response if the file doesn't exist. The code that calls that function should check for and interpret that condition. ---------------------------------------------------------------------- Comment By: SHIGENO Kazutaka (shigeno) Date: 2004-10-15 00:11 Message: Logged In: YES user_id=1138453 How about changing as follows about self.__releasetime()? @@ -448,7 +448,7 @@ return os.stat(self.__lockfile)[ST_MTIME] except OSError, e: if e.errno <> errno.ENOENT: raise - return -1 + return time.time() def __linkcount(self): try: ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_…

1 0

[ mailman-Patches-1045656 ] Race in LockFile.py resulting in extraneous log entries.
by SourceForge.net 24 Oct '04

24 Oct '04

Patches item #1045656, was opened at 2004-10-12 19:50 Message generated for change (Comment added) made by ppsys You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_… Category: mail delivery Group: Mailman 2.1 Status: Open Resolution: None Priority: 3 Submitted By: Brian Greenberg (grnbrg) Assigned to: Tokio Kikuchi (tkikuchi) Summary: Race in LockFile.py resulting in extraneous log entries. Initial Comment: There is a race condition (that does not affect correct operation of Mailman) in .../Mailman/LockFile.py. This race results in entries to the "locks" log file (usually in pairs) that are unnecessary and confusing. If there is a process holding a lock for a file, and a process waiting for that lock to be freed, the following sequence can occur: 1) The waiting process executes os.link(). Since the running process still has the lock, this fails with EEXIST. 2) The running process releases the lock, and removes the lock file. 3) The waiting process proceeds with it's checks, and checks self.__linkcount() which returns -1 due to an ENOENT error. This throws an error into the "locks" log file. 4) The waiting process continues processing, checks it's timeout, and then checks to see if the lock lifetime has expired. The lifetime check is based on a call to self.__releasetime(), which returns -1, due to an ENOENT error. This results in the lockfile being declared stale, and self.__break() is called. This then throws a second error into the "locks" log file. The attached patch checks for these ENOENT errors, immediately restarting the loop when they are detected. It does not affect operation is the lock file exists and is held by another process. Brian Greenberg. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2004-10-24 08:02 Message: Logged In: YES user_id=75166 The comments in the LockFile.py header code make the point that the entire mechanism is to deal with NFS robustness/compatibilty issues. The extranenous logging that concerns grnbrg is appearing on Solaris (version and build unspecified) and attempts to reproduce it, that have thus far failed, appear to be on FreeBSD. I suspect that the majority of MM installs are on various versions and builds of Linux. These OSen will all have substantially different NFS implementations and major differences in their kernel that probably affect NFS implementation and operation. Because of these differences I have already encountered sufficient problems with interworking of Linux NFS clients with Solaris NFS servers to convince me that changing LockFile.py without sufficient testing in order to clear an essentially cosmetic problem on one host OS is a an unecessary risk to stability across the board. Just testing on one host is not good enough and reading code even less so; it is the dynamics of operation in live situations that are the issue. A test programme needs to at least nod in the direction of heterogenous NFS client/host configurations of various types being in used for validation and verification of any changes. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-22 18:53 Message: Logged In: YES user_id=902583 I'm running on a Solaris box, which may or may not have any impact on the issue. I'm not sure how to force reproduction -- races are hard to intentially duplicate. I first noticed the problem under a heavy load due to an admin "oops" -- 30,000+ news articles being gatewayed into a test list. :) Once the oops was fixed, the error logs continued to occasionally appear, and rather than assume that everything was ok, I tracked down the source of the problem. I did this by desk-checking the source, and identified the race by a bit of guesswork. Once these changes were made, the problem stopped. The race occurs as follows: 1) A qrunner runs LockFile.lock() to request a particular lock. 2) At LockFile.py:256, this qrunner attempts to link it's temp lock file to the global lockfile. This link fails with a EEXIST error, because another process currently holds the lock. The assumption is made at this point that the global lockfile exists. This (false) assumption continues through to the end of the loop at LockFile.py:312. In fact, the global lockfile may cease to exist (because the qrunner that has it calls unlock()) at any time. 3) The process holding the lock calls unlock(). 4) The waiting process (which just got an EEXIST) goes through it's various sanity checks, which include checking the linkcount and mtime of the global lockfile, which now does not exist. Since it is assumed to exist, the -1 results returned make no sense, and are unconditionally logged. 5) At LockFile.py:303, the (incorrectly assumed) existing lock is broken, and control goes back to the top of the loop (after a quick __sleep()) As far as impact, the patch should have minimal side effects. I pulled the function calls out of the if statements, so there is no change in the number of calls. Further, all the new code does is to check for the "-1" return that indicates that the global lockfile no longer exists, and immediately restarts the loop if that is the case. It might not be a bad idea to add a "self.__sleep()" before the continue, but I didn't think it was needed. Hope that adds enough detail to trace what I think is happening, and verify that it won't break anything. FWIW, I have not had any problems, nor any errors logged since applying this locally. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2004-10-22 12:06 Message: Logged In: YES user_id=12800 I haven't had time to look at this in detail, but I'll just include a general warning that LockFile.py is /very/ fragile. It's also critical to the proper operation of Mailman. So any changes, even seemingly innocent onces, must be very thoroughly tested. My inclination would be to reject any change that wasn't absolutely necessary, but I won't veto this change if it is well-tested and fixes a verified bug (tho' it sounds like tkikuchi can't reproduce it). ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 08:26 Message: Logged In: YES user_id=67709 Will you be more specific how this race occurs? My installation on FreeBSD doesn't looks like show this symptom in locks log file. I tested with two terminals invoking python -i bin/withlist testlist and try to lock with m.Lock() in both the terminal. One of term is forced to wait until the I type m.Unlock() in the other term. I then looked into logs dir but nothing avail. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-15 14:22 Message: Logged In: YES user_id=902583 This would fix this bug, as long as CLOCK_SLOP is large enough. But I don't think it's the Right Thing, as you are returning a result that says that the lockfile has a lifetime, when it in fact simply no longer exists. For a function whose purpose is to report the mtime of the master lockfile, -1 is the correct response if the file doesn't exist. The code that calls that function should check for and interpret that condition. ---------------------------------------------------------------------- Comment By: SHIGENO Kazutaka (shigeno) Date: 2004-10-15 00:11 Message: Logged In: YES user_id=1138453 How about changing as follows about self.__releasetime()? @@ -448,7 +448,7 @@ return os.stat(self.__lockfile)[ST_MTIME] except OSError, e: if e.errno <> errno.ENOENT: raise - return -1 + return time.time() def __linkcount(self): try: ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_…

1 0

[ mailman-Patches-1045656 ] Race in LockFile.py resulting in extraneous log entries.
by SourceForge.net 22 Oct '04

22 Oct '04

Patches item #1045656, was opened at 2004-10-12 14:50 Message generated for change (Comment added) made by grnbrg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_… Category: mail delivery Group: Mailman 2.1 Status: Open Resolution: None Priority: 3 Submitted By: Brian Greenberg (grnbrg) Assigned to: Tokio Kikuchi (tkikuchi) Summary: Race in LockFile.py resulting in extraneous log entries. Initial Comment: There is a race condition (that does not affect correct operation of Mailman) in .../Mailman/LockFile.py. This race results in entries to the "locks" log file (usually in pairs) that are unnecessary and confusing. If there is a process holding a lock for a file, and a process waiting for that lock to be freed, the following sequence can occur: 1) The waiting process executes os.link(). Since the running process still has the lock, this fails with EEXIST. 2) The running process releases the lock, and removes the lock file. 3) The waiting process proceeds with it's checks, and checks self.__linkcount() which returns -1 due to an ENOENT error. This throws an error into the "locks" log file. 4) The waiting process continues processing, checks it's timeout, and then checks to see if the lock lifetime has expired. The lifetime check is based on a call to self.__releasetime(), which returns -1, due to an ENOENT error. This results in the lockfile being declared stale, and self.__break() is called. This then throws a second error into the "locks" log file. The attached patch checks for these ENOENT errors, immediately restarting the loop when they are detected. It does not affect operation is the lock file exists and is held by another process. Brian Greenberg. ---------------------------------------------------------------------- >Comment By: Brian Greenberg (grnbrg) Date: 2004-10-22 13:53 Message: Logged In: YES user_id=902583 I'm running on a Solaris box, which may or may not have any impact on the issue. I'm not sure how to force reproduction -- races are hard to intentially duplicate. I first noticed the problem under a heavy load due to an admin "oops" -- 30,000+ news articles being gatewayed into a test list. :) Once the oops was fixed, the error logs continued to occasionally appear, and rather than assume that everything was ok, I tracked down the source of the problem. I did this by desk-checking the source, and identified the race by a bit of guesswork. Once these changes were made, the problem stopped. The race occurs as follows: 1) A qrunner runs LockFile.lock() to request a particular lock. 2) At LockFile.py:256, this qrunner attempts to link it's temp lock file to the global lockfile. This link fails with a EEXIST error, because another process currently holds the lock. The assumption is made at this point that the global lockfile exists. This (false) assumption continues through to the end of the loop at LockFile.py:312. In fact, the global lockfile may cease to exist (because the qrunner that has it calls unlock()) at any time. 3) The process holding the lock calls unlock(). 4) The waiting process (which just got an EEXIST) goes through it's various sanity checks, which include checking the linkcount and mtime of the global lockfile, which now does not exist. Since it is assumed to exist, the -1 results returned make no sense, and are unconditionally logged. 5) At LockFile.py:303, the (incorrectly assumed) existing lock is broken, and control goes back to the top of the loop (after a quick __sleep()) As far as impact, the patch should have minimal side effects. I pulled the function calls out of the if statements, so there is no change in the number of calls. Further, all the new code does is to check for the "-1" return that indicates that the global lockfile no longer exists, and immediately restarts the loop if that is the case. It might not be a bad idea to add a "self.__sleep()" before the continue, but I didn't think it was needed. Hope that adds enough detail to trace what I think is happening, and verify that it won't break anything. FWIW, I have not had any problems, nor any errors logged since applying this locally. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2004-10-22 07:06 Message: Logged In: YES user_id=12800 I haven't had time to look at this in detail, but I'll just include a general warning that LockFile.py is /very/ fragile. It's also critical to the proper operation of Mailman. So any changes, even seemingly innocent onces, must be very thoroughly tested. My inclination would be to reject any change that wasn't absolutely necessary, but I won't veto this change if it is well-tested and fixes a verified bug (tho' it sounds like tkikuchi can't reproduce it). ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 03:26 Message: Logged In: YES user_id=67709 Will you be more specific how this race occurs? My installation on FreeBSD doesn't looks like show this symptom in locks log file. I tested with two terminals invoking python -i bin/withlist testlist and try to lock with m.Lock() in both the terminal. One of term is forced to wait until the I type m.Unlock() in the other term. I then looked into logs dir but nothing avail. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-15 09:22 Message: Logged In: YES user_id=902583 This would fix this bug, as long as CLOCK_SLOP is large enough. But I don't think it's the Right Thing, as you are returning a result that says that the lockfile has a lifetime, when it in fact simply no longer exists. For a function whose purpose is to report the mtime of the master lockfile, -1 is the correct response if the file doesn't exist. The code that calls that function should check for and interpret that condition. ---------------------------------------------------------------------- Comment By: SHIGENO Kazutaka (shigeno) Date: 2004-10-14 19:11 Message: Logged In: YES user_id=1138453 How about changing as follows about self.__releasetime()? @@ -448,7 +448,7 @@ return os.stat(self.__lockfile)[ST_MTIME] except OSError, e: if e.errno <> errno.ENOENT: raise - return -1 + return time.time() def __linkcount(self): try: ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_…

1 0

[ mailman-Bugs-1050878 ] "Cancel my subscription request" error
by SourceForge.net 22 Oct '04

22 Oct '04

Bugs item #1050878, was opened at 2004-10-20 16:49 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1050878&group_… Category: (un)subscribing Group: 2.1 (stable) >Status: Closed Resolution: None Priority: 5 Submitted By: cmika (cmika) Assigned to: Nobody/Anonymous (nobody) Summary: "Cancel my subscription request" error Initial Comment: Bug in Mailman version 2.1.5 We're sorry, we hit a bug! If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! Traceback: Traceback (most recent call last): File "/usr/local/lib/mailman/scripts/driver", line 87, in run_main main() File "/usr/obj/ports/mailman-2.1.5/fake-i386/usr/local/lib/mailman/Mailman/Cgi/confirm.py", line 114, in main File "/usr/obj/ports/mailman-2.1.5/fake-i386/usr/local/lib/mailman/Mailman/Cgi/confirm.py", line 312, in subscription_cancel File "/usr/obj/ports/mailman-2.1.5/fake-i386/usr/local/lib/mailman/Mailman/Pending.py", line 141, in pend_confirm AssertionError Python information: Variable Value sys.version 2.2.3 (#1, Mar 23 2004, 19:11:01) [GCC 2.95.3 20010125 (prerelease, propolice)] sys.executable /usr/local/bin/python2.2 sys.prefix /usr/local sys.exec_prefix /usr/local sys.path /usr/local sys.platform openbsd3 Environment variables: Variable Value PATH_INFO /friends HTTP_ACCEPT text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 CONTENT_TYPE application/x-www-form-urlencoded HTTP_REFERER https://lists.seenothing.org/confirm/friends/26ebbe661ef86491452776e04efd2a… SERVER_SOFTWARE Apache/1.3.29 (Unix) PHP/4.3.8 mod_ssl/2.8.16 OpenSSL/0.9.7c PYTHONPATH /usr/local/lib/mailman SCRIPT_FILENAME /usr/local/lib/mailman/cgi-bin/confirm SERVER_ADMIN webmaster(a)lists.seenothing.org SCRIPT_NAME /confirm SERVER_SIGNATURE Apache/1.3.29 Server at lists.seenothing.org Port 443 REQUEST_METHOD POST HTTP_HOST lists.seenothing.org HTTP_KEEP_ALIVE 300 SERVER_PROTOCOL HTTP/1.1 QUERY_STRING REQUEST_URI /confirm/friends CONTENT_LENGTH 117 HTTP_ACCEPT_CHARSET ISO-8859-1,utf-8;q=0.7,*;q=0.7 HTTP_USER_AGENT Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.2) Gecko/20040809 Galeon/1.3.17 HTTP_CONNECTION keep-alive HTTP_COOKIE friends+user+cmika--at--seenothing.org=2802000000696b957641732800000038323862613031396536373461656238326664646166363064643930623331666538626361613262 SERVER_NAME lists.seenothing.org REMOTE_ADDR 66.80.107.196 REMOTE_PORT 60843 HTTP_ACCEPT_LANGUAGE en PATH_TRANSLATED /usr/local/lib/mailman/cgi-bin/friends SERVER_PORT 443 GATEWAY_INTERFACE CGI/1.1 HTTP_ACCEPT_ENCODING gzip,deflate SERVER_ADDR 192.168.1.2 DOCUMENT_ROOT /var/www/htdocs When a user tries to cancel subscription request, mailman comes up with an error. Mailman's installed on an OpenBSD 3.5 machine, using packages. ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 13:02 Message: Logged In: YES user_id=67709 This was fixed in CVS. See closed item #981188 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1050878&group_…

1 0

[ mailman-Patches-1045656 ] Race in LockFile.py resulting in extraneous log entries.
by SourceForge.net 22 Oct '04

22 Oct '04

Patches item #1045656, was opened at 2004-10-12 15:50 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_… Category: mail delivery Group: Mailman 2.1 Status: Open Resolution: None Priority: 3 Submitted By: Brian Greenberg (grnbrg) Assigned to: Tokio Kikuchi (tkikuchi) Summary: Race in LockFile.py resulting in extraneous log entries. Initial Comment: There is a race condition (that does not affect correct operation of Mailman) in .../Mailman/LockFile.py. This race results in entries to the "locks" log file (usually in pairs) that are unnecessary and confusing. If there is a process holding a lock for a file, and a process waiting for that lock to be freed, the following sequence can occur: 1) The waiting process executes os.link(). Since the running process still has the lock, this fails with EEXIST. 2) The running process releases the lock, and removes the lock file. 3) The waiting process proceeds with it's checks, and checks self.__linkcount() which returns -1 due to an ENOENT error. This throws an error into the "locks" log file. 4) The waiting process continues processing, checks it's timeout, and then checks to see if the lock lifetime has expired. The lifetime check is based on a call to self.__releasetime(), which returns -1, due to an ENOENT error. This results in the lockfile being declared stale, and self.__break() is called. This then throws a second error into the "locks" log file. The attached patch checks for these ENOENT errors, immediately restarting the loop when they are detected. It does not affect operation is the lock file exists and is held by another process. Brian Greenberg. ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2004-10-22 08:06 Message: Logged In: YES user_id=12800 I haven't had time to look at this in detail, but I'll just include a general warning that LockFile.py is /very/ fragile. It's also critical to the proper operation of Mailman. So any changes, even seemingly innocent onces, must be very thoroughly tested. My inclination would be to reject any change that wasn't absolutely necessary, but I won't veto this change if it is well-tested and fixes a verified bug (tho' it sounds like tkikuchi can't reproduce it). ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-10-22 04:26 Message: Logged In: YES user_id=67709 Will you be more specific how this race occurs? My installation on FreeBSD doesn't looks like show this symptom in locks log file. I tested with two terminals invoking python -i bin/withlist testlist and try to lock with m.Lock() in both the terminal. One of term is forced to wait until the I type m.Unlock() in the other term. I then looked into logs dir but nothing avail. ---------------------------------------------------------------------- Comment By: Brian Greenberg (grnbrg) Date: 2004-10-15 10:22 Message: Logged In: YES user_id=902583 This would fix this bug, as long as CLOCK_SLOP is large enough. But I don't think it's the Right Thing, as you are returning a result that says that the lockfile has a lifetime, when it in fact simply no longer exists. For a function whose purpose is to report the mtime of the master lockfile, -1 is the correct response if the file doesn't exist. The code that calls that function should check for and interpret that condition. ---------------------------------------------------------------------- Comment By: SHIGENO Kazutaka (shigeno) Date: 2004-10-14 20:11 Message: Logged In: YES user_id=1138453 How about changing as follows about self.__releasetime()? @@ -448,7 +448,7 @@ return os.stat(self.__lockfile)[ST_MTIME] except OSError, e: if e.errno <> errno.ENOENT: raise - return -1 + return time.time() def __linkcount(self): try: ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1045656&group_…

1 0