the release gods are angry at python
The next releases of 2.6/3.0 are planned for April 2, just over a week from now. There is much work that needs to be done. The buildbots are in a pretty sad state and the gods are seeing too much red. http://www.python.org/dev/buildbot/stable/ http://www.python.org/dev/buildbot/all/ See my other mail that discusses the stable buildbots. The criteria for release is that all the stable buildbots are passing all the tests. So we really gotta get these green before Barry notices. You don't want to see Barry angry. You wouldn't like him when he's angry. I propose a code chill for new features. Changes to doc and tests can continue as usual. However, only submit a new feature *after* you fix a broken test first. If we have to get draconian, we can start breaking fingers when you break a test just like we do at work. :-) Specifically tests that need some TLC are: * test_winsound - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/ste... * test_threading - test_no_refcycle_through_target - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/ste... * test_socket deadlocks and times out - http://www.python.org/dev/buildbot/stable/x86%20W2k8%20trunk/builds/210/step... * test_ssl deadlocks and times out - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/ste... * test_xmlrpc transient socket errors - http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/ste... * test_mailbox - http://www.python.org/dev/buildbot/stable/x86%20XP-3%203.0/builds/723/step-t... * test_asynchat - http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/27... Hopefully test_timeout is fixed, but that might be flaky too. There have been other tests that have also been flaky like test_asynchat, test_smtplib, test_ssl, test_urllib2net, test_urllibnet, test_xmlrpc_net and some of the tests that use networking. These all need to be fixed so the tests are 100% reliable and only fail when there is a real error. There are currently no release blocker issues: http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=1&%40group=priority&status=1&%40action=search There are 48 critical issues: http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=2&%40group=priority&status=1&%40action=search If you believe any issue should block the release, set the priority to release blocker and assign it to me (nnorwitz). Many of the critical issues are those that require 2to3 fixers. These can be checked in as they are written. Be sure to test them thoroughly and try to think of all the conditions that could possibly cause the fixer to fail or do the wrong thing. Right now, I don't know of any reason to hold up the release other than the failing tests. n
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2008, at 2:21 AM, Neal Norwitz wrote:
The next releases of 2.6/3.0 are planned for April 2, just over a week from now. There is much work that needs to be done. The buildbots are in a pretty sad state and the gods are seeing too much red.
http://www.python.org/dev/buildbot/stable/ http://www.python.org/dev/buildbot/all/
See my other mail that discusses the stable buildbots. The criteria for release is that all the stable buildbots are passing all the tests. So we really gotta get these green before Barry notices. You don't want to see Barry angry. You wouldn't like him when he's angry.
Of course, most people don't like me when I'm /not/ angry either! :) Thanks for being the Bad Cop on this Neal. Please folks, please help make these buildbots go green! I think we should all be disappointed if the releases have to slip because our stable buildbots are red. Neal and I will have free rein to back out changes if that turns them green so if your code changes cause a failure, and you want your changes to stay in, please spend some time fixing them.
I propose a code chill for new features. Changes to doc and tests can continue as usual. However, only submit a new feature *after* you fix a broken test first.
+1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR+pt4XEjvBPtnXfVAQKHQwP/WXCoUtJY7nq3xFExs5RCCEeWFceSBRJR XBjymIVVODaWyhzh2obCuD/reHiEHneVWBzrS+kuQPEdigR6R4wJIyLQNqol5bfD auDJzUAa4vrMjfJtf2+i3/GRaHPHzgwPfgyj1t5rlyE/3pLbSh+d4+qhtH8Hq0MY ExF6WVIWRDc= =FP2J -----END PGP SIGNATURE-----
On Wed, Mar 26, 2008 at 1:21 AM, Neal Norwitz
The next releases of 2.6/3.0 are planned for April 2, just over a week from now. There is much work that needs to be done. The buildbots are in a pretty sad state and the gods are seeing too much red.
http://www.python.org/dev/buildbot/stable/ http://www.python.org/dev/buildbot/all/
See my other mail that discusses the stable buildbots. The criteria for release is that all the stable buildbots are passing all the tests. So we really gotta get these green before Barry notices. You don't want to see Barry angry. You wouldn't like him when he's angry.
I propose a code chill for new features. Changes to doc and tests can continue as usual. However, only submit a new feature *after* you fix a broken test first. If we have to get draconian, we can start breaking fingers when you break a test just like we do at work. :-)
Specifically tests that need some TLC are: * test_winsound - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/ste... * test_threading - test_no_refcycle_through_target - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/ste... * test_socket deadlocks and times out - http://www.python.org/dev/buildbot/stable/x86%20W2k8%20trunk/builds/210/step... * test_ssl deadlocks and times out - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/ste... * test_xmlrpc transient socket errors - http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/ste... * test_mailbox - http://www.python.org/dev/buildbot/stable/x86%20XP-3%203.0/builds/723/step-t... * test_asynchat - http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/27...
Hopefully test_timeout is fixed, but that might be flaky too. There have been other tests that have also been flaky like test_asynchat, test_smtplib, test_ssl, test_urllib2net, test_urllibnet, test_xmlrpc_net and some of the tests that use networking. These all need to be fixed so the tests are 100% reliable and only fail when there is a real error.
There are currently no release blocker issues:
There are 48 critical issues:
If you believe any issue should block the release, set the priority to release blocker and assign it to me (nnorwitz). Many of the critical issues are those that require 2to3 fixers. These can be checked in as they are written. Be sure to test them thoroughly and try to think of all the conditions that could possibly cause the fixer to fail or do the wrong thing.
There are also some backporting issues in that pile. Should those hold up betas? (when we get there)
Right now, I don't know of any reason to hold up the release other than the failing tests.
n _______________________________________________ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/musiccomposition%40gmail....
-- Cheers, Benjamin Peterson
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2008, at 5:10 PM, Benjamin Peterson wrote:
There are also some backporting issues in that pile. Should those hold up betas? (when we get there)
Yes, but I would simply release the monthly alpha and push the beta back a month. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+q/dXEjvBPtnXfVAQJYJAP9En+87NRtSOKcnEB4Om/BYE+Jw7KZ08JI HY1MGAHleya8MHIgaEkaQf142pzfHKWxWZLNWD5UShHPlarwfvIxHdT07q5ozGda l4J95FU2EaLMGxN+5HvxlY+pdXAiVcNAnO12TO1Gqahf9Mmk1eXkKM0p6vBPJHqZ vtekrWIzeck= =3v/o -----END PGP SIGNATURE-----
>> The next releases of 2.6/3.0 are planned for April 2, just over a >> week from now. There is much work that needs to be done. The >> buildbots are in a pretty sad state and the gods are seeing too much >> red. >> >> http://www.python.org/dev/buildbot/stable/ >> http://www.python.org/dev/buildbot/all/ Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I checked it this morning it was idle with two pending. I forced a run. I checked later today and it showed five pending. Now it shows 10 pending. It looks like its last run was about 18 hours ago. You would think by now it would have been able to make another run. It pings fine. Skip
There have been other tests that have also been flaky like test_asynchat, test_smtplib, test_ssl, test_urllib2net, test_urllibnet, test_xmlrpc_net and some of the tests that use networking.
Some of the *other* tests that use networking, I think you mean. Sounds like networking tests in general are flaky.
- http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/ste...
Unfortunately, this log has no information about why the test is failing, and I do not have a Debian-unstable machine to try it on (much less a six-processor IBM S/390 mainframe running Debian-unstable -- cool!). I'm unsure about how to make progress here. It's interesting to see that most of the stable buildbots running Debian are doing so on big-endian (PPC, Sparc) hardware. I also can't tell which branch of Python is being tested, from this logfile. That would be nice to add. Is this 2.6 (known problems with SSL) or 3.0 (no known problems with SSL)? Is this information in the logfile somewhere and I'm not seeing it? Bill
Bill Janssen wrote:
There have been other tests that have also been flaky like test_asynchat, test_smtplib, test_ssl, test_urllib2net, test_urllibnet, test_xmlrpc_net and some of the tests that use networking.
Some of the *other* tests that use networking, I think you mean. Sounds like networking tests in general are flaky.
This patch was put together at PyCon (but has not been applied): http://bugs.python.org/issue2429 It moves some of the urllib2net tests to urllib2localnet - they use a local server instead of going out to external servers and should be more reliable. Michael
- http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/ste...
Unfortunately, this log has no information about why the test is failing, and I do not have a Debian-unstable machine to try it on (much less a six-processor IBM S/390 mainframe running Debian-unstable -- cool!). I'm unsure about how to make progress here. It's interesting to see that most of the stable buildbots running Debian are doing so on big-endian (PPC, Sparc) hardware.
I also can't tell which branch of Python is being tested, from this logfile. That would be nice to add. Is this 2.6 (known problems with SSL) or 3.0 (no known problems with SSL)? Is this information in the logfile somewhere and I'm not seeing it?
Bill _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On Wed, Mar 26, 2008 at 5:52 PM,
>> The next releases of 2.6/3.0 are planned for April 2, just over a >> week from now. There is much work that needs to be done. The >> buildbots are in a pretty sad state and the gods are seeing too much >> red. >> >> http://www.python.org/dev/buildbot/stable/ >> http://www.python.org/dev/buildbot/all/
Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I checked it this morning it was idle with two pending. I forced a run. I checked later today and it showed five pending. Now it shows 10 pending. It looks like its last run was about 18 hours ago. You would think by now it would have been able to make another run. It pings fine.
I logged in to the machine an the buildbot is running. I think what happens is that sometimes the master loses track of the slaves. The fix requires restarting the master, but I didn't want to do that last night while there were still builds going on. I'll try to find a quiet time tonight and restart the master. I expect that will fix the problem. n
On Thu, 27 Mar 2008 11:43:25 -0700, Neal Norwitz
On Wed, Mar 26, 2008 at 5:52 PM,
wrote: >> The next releases of 2.6/3.0 are planned for April 2, just over a >> week from now. There is much work that needs to be done. The >> buildbots are in a pretty sad state and the gods are seeing too much >> red. >> >> http://www.python.org/dev/buildbot/stable/ >> http://www.python.org/dev/buildbot/all/
Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I checked it this morning it was idle with two pending. I forced a run. I checked later today and it showed five pending. Now it shows 10 pending. It looks like its last run was about 18 hours ago. You would think by now it would have been able to make another run. It pings fine.
I logged in to the machine an the buildbot is running. I think what happens is that sometimes the master loses track of the slaves. The fix requires restarting the master, but I didn't want to do that last night while there were still builds going on. I'll try to find a quiet time tonight and restart the master. I expect that will fix the problem.
There's a bug in some configurations (I have never managed to track down the details) where the ping action actually prevents any further builds from happening on that slave until the master is restarted. Not sure if this is related to the problem you're seeing here or not. Jean-Paul
Jean-Paul Calderone
There's a bug in some configurations (I have never managed to track down the details) where the ping action actually prevents any further builds from happening on that slave until the master is restarted. Not sure if this is related to the problem you're seeing here or not.
Over time, I've seen this occasionally in the status for my buildbots (idle, but builds shown as pending, and yes often the last action was a ping). At least in the cases I've observed into so far, restarting the buildbot has typically cleaned it up and let the builds proceed. -- David
On Wed, Mar 26, 2008 at 7:21 AM, Neal Norwitz
* test_xmlrpc transient socket errors - http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/ste...
These are caused by the accept call returning a nonblocking socket, when the listening socket itself is nonblocking (which it is). see http://bugs.python.org/issue1503 for more details.
participants (9)
-
Barry Warsaw
-
Benjamin Peterson
-
Bill Janssen
-
David Bolen
-
Jean-Paul Calderone
-
Michael Foord
-
Neal Norwitz
-
Ralf Schmitt
-
skip@pobox.com