Python 2.3.4c1 -- nasty threading bug (Linux) [resend]

[Anthony Baxter]

[this is a resend - thunderbird crashed the first time I sent it]

Dieter Maurer wrote:
 >>The patch wasn't applied, and the 2.3.4 release manager doesn't want 
to risk
 >>it for 2.3.4.  Do read the bug report!  There are 36 comments 
attached (so
 >>far) because it's a complex problem to solve it correctly.  Andrew 
Langmead
 >>(whose patch you referenced) is continuing to participate in finding a
 >>correct solution.  See his comment there dated 2004-05-04 10:00 for the
 >>relevant respect in which LinuxThreads is buggy but NPTL is not.
 >
 >
 > I verified that the patch mentioned above indeed fixes the problem.
 >
 > It might introduce other subtle problems but at least none that
 > are revealed by Python's regression test suite...
 >
 > Moreover, I doubt that such problems will be significant in practise:
 >
 >   The patch prevents blocking of signals that should (as specified by the
 >   pthreads standard)
 >   not be blocked -- as the operating system uses these signals
 >   to report serious problems. No application should use
 >   these threads for application specific communication.
 >
 >   Therefore, a violation of Python's principle to only
 >   deliver signals to the main thread seems appropriate
 >   for these signals.
 >
 >
 > It would be great, when the Python developers could come up with
 > an even better patch. However, those who run "productive"
 > multithreaded Python based applications, especially Zope 2,
 > on Linux with LinuxThreads (rather than native PosixThreads)
 > may consider to use the patch which is available *now*.
 >
 > We will do so. I will report back should we see more serious
 > problems than we have now.

Just as a followup to explain why this wasn't in 2.3.4...

This patch, as at the current time, _still_ has not been applied
to the trunk. This is now a week and a half after the initial 2.3.4
final release date was planned. There are interesting (ha!) issues
relating to how it works with readline, it's still unknown about any
portability issues that it might cause, and in general it's a huge can
of worms. I'm glad that it fixes the case you've had with Zope, on the
platform you tried it on, but that is _not_ enough to say "whack it into
a bug fix release". Zope is not the only Python application out there,
and Linux is not the only platform.

In general, the maintenance branch gets fixes from the trunk, after
they've been tried out and found to be good (particularly in this
case, where the possible problems are likely to be subtle and fiendish
to track down.)

The goal is that this fix will be nailed down and in the first alpha
of 2.4. We can then beat on this during the 2.4 release cycle and verify
that the fix is good. At _that_ point, it will be backported to the 2.3
branch, and will be in 2.3.5 (which will be released not-too-long
after 2.4 final, and will probably be the last of the 2.3 series).

My absolute #1 goal here is to produce high quality bug fix releases
that people can install with confidence that it won't introduce ugly
new regressions. This particular patch does _not_ come close at the
current time. Yes, this bug is annoying for those people experiencing
it, but it's nowhere near critical enough to hold off on a release.
Particularly when, as I've mentioned, there still isn't a final fix.

Note also that for all future releases of Python, both bugfix and
major, I will be sending out a 'pre-announcement' a month before
the release date to try and shake these bugs out of the trees
(a message was sent out a day or so ago for 2.4a1). At the moment,
everyone waits until the release candidate is cut to push for
their bugfix of choice - this is not a path that leads to either
solid releases or a less-stressed release manager.

I hope this helps people understand some of the release process...

Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.