Re: [Python-Dev] Py3k and asyncore/asynchat
On Tue, Mar 25, 2008 at 11:26 PM, Neal Norwitz
Any reason this was sent just to me and not the list?
Because gmail only replies to the sender by default. I need to remember to cc python-dev when I reply (I used the same email client for 8 1/2 years, remembering the quirks of gmail may take some time).
On Tue, Mar 25, 2008 at 10:34 PM, Josiah Carlson
wrote: On Tue, Mar 25, 2008 at 9:00 PM, Neal Norwitz
wrote: On Thu, Feb 14, 2008 at 10:09 AM, Giampaolo Rodola'
wrote: On 14 Feb, 16:36, "Giampaolo Rodola'"
wrote: Ok, I'll try to take a look at all asyncore/chat reports and try to summarize them by splitting patches which solve bugs and patches which add enhancements or functionalities.
=== Patches for existing issues ===
- 1736190 which includes fixes for the following issues among other improvements: - 1063924 (asyncore should handle ECONNRESET in send()) - 1736101 (asyncore should handle ECONNABORTED in recv()) - 760475 (handle_error() should call handle_close() instead of close()) - 1740572 (refill_buffer() should call handle_close() rather than close()) - 777588 (wrong "connection refused" behavior on Windows) - 889153 (wrong connect() behavior) - 953599 (asyncore misses socket closes when poll is used) - 1025525 (asyncore.file_dispatcher should not take fd as argument)
- 1519 (async_chat.__init__() and asyncore.dispatcher.__init__ parameters inconsistency) - 1541 (Bad OOB data management when using asyncore with select.poll()) - 2073 (asynchat push always sends 512 bytes (ignoring ac_out_buffer_size))
=== Open issues with no patches (need review) ===
- 658749 (asyncore connect() and winsock errors) - 1161031 (neverending warnings from asyncore)
=== Enhancements & new features ===
- 1641 (add delayed calls feature) - 1563 (conversion to py3k and some other changes)
That's a lot of patches. My immediate concern is that test_asynchat is very flaky and fails often. Sometimes it causes other tests to fail. Is there a patch that addresses this? If you need examples, just look through the buildbot mails that mention test_asynchat in: http://mail.python.org/pipermail/python-checkins/
No, it's one patch. All of the fixes were performed mostly by myself last spring, tested and verified in personal servers, then re-used by Giampaolo in his async ftp server (which pointed out a few small bugs, which have been fixed).
Some platforms have more problems than others, but almost all platforms have failed test_asynchat at one point or another.
Certainly that is the case. And according to my reading of a few buildbot failures, aside from someone breaking asyncore itself, the failures seem to stem from the test being unable to create a port to listen on in order to test the server/client functionality. This is a buildbot configuration issue (per host), not an asyncore issue.
That was the case a long time (~3? months) ago, but hasn't been the case recently. See my recent message about the release.
I'll look for it tomorrow. For reference, searches of 'site:mail.python.org test_asynchat failure buildbot' only seem to produce the socket listen error. If there is a better incantation to get google to produce the proper errors (and/or a link), I would appreciate the help.
Please break up the patches into 2 sets and prioritize the patches with the set.
Set #1: Patches that have a test and doc updates if required Set #2: Patches that don't have a test or doc updates as required
For the patches that fall into Set #1, list them in priority order. Top priority would be a problem that fixes failures seen in the buildbots. Next priority would go to the patches that solve more serious problems. Post the results here. I will try to look at them.
For every patch you list, make sure that it conforms to the proper style (e.g, PEP 8) and is essentially perfect and ready for inclusion. This means that there is a single file to download that contains all the modifications. The changes are appropriately commented, lines are less than 80 characters, etc. One of the modifications should be an entry in Misc/NEWS.
I lied earlier; really there are two patches. The first is a patch to asyncore.py and asynchat.py . It addresses those bugs that Giampaolo has listed, it is tested, and works. The second patch is to update the documentation to mention the sample methods in asynchat for use as examples, as well as any other changes to the language used in the documentation that I had made last spring, but which are out of date from my posting of the original patch. I can update the documentation in the next week.
Can you provide a link to the patches? Do the patches include changes to test_asyncore and test_asynchat? The next release is April 2. I would like to commit any patches before Monday to ensure they are stable. Can you get me the patches by this Saturday?
See http://bugs.python.org/issue1736190 for an updated patch for the modules. The current test cases pass without issue, though we may want to add tests, which I need to look at the original patch and the original file from which it was created against, then compare it with the most recent changes to the tests from Facundo last June or July. I should have the time to get patches for tests and documentation by Monday. - Josiah
(sorry for top posting)
I haven't really had time to update the tests/documentation, but
again, I wasn't experiencing any strange test failures with
asyncore/asynchat, nor have I been able to find the buildbot failures
that you are referring to. Could someone please link the failures
that are not related to being unable to discover a port number?
According to the release schedule, we should have at least a couple
more months for documentation and tests to be updated (I can get
patches ready for alpha 3).
- Josiah
On Wed, Mar 26, 2008 at 12:21 AM, Josiah Carlson
On Tue, Mar 25, 2008 at 11:26 PM, Neal Norwitz
wrote: Any reason this was sent just to me and not the list?
Because gmail only replies to the sender by default. I need to remember to cc python-dev when I reply (I used the same email client for 8 1/2 years, remembering the quirks of gmail may take some time).
On Tue, Mar 25, 2008 at 10:34 PM, Josiah Carlson
wrote: On Tue, Mar 25, 2008 at 9:00 PM, Neal Norwitz
wrote: On Thu, Feb 14, 2008 at 10:09 AM, Giampaolo Rodola'
wrote: On 14 Feb, 16:36, "Giampaolo Rodola'"
wrote: Ok, I'll try to take a look at all asyncore/chat reports and try to summarize them by splitting patches which solve bugs and patches which add enhancements or functionalities.
=== Patches for existing issues ===
- 1736190 which includes fixes for the following issues among other improvements: - 1063924 (asyncore should handle ECONNRESET in send()) - 1736101 (asyncore should handle ECONNABORTED in recv()) - 760475 (handle_error() should call handle_close() instead of close()) - 1740572 (refill_buffer() should call handle_close() rather than close()) - 777588 (wrong "connection refused" behavior on Windows) - 889153 (wrong connect() behavior) - 953599 (asyncore misses socket closes when poll is used) - 1025525 (asyncore.file_dispatcher should not take fd as argument)
- 1519 (async_chat.__init__() and asyncore.dispatcher.__init__ parameters inconsistency) - 1541 (Bad OOB data management when using asyncore with select.poll()) - 2073 (asynchat push always sends 512 bytes (ignoring ac_out_buffer_size))
=== Open issues with no patches (need review) ===
- 658749 (asyncore connect() and winsock errors) - 1161031 (neverending warnings from asyncore)
=== Enhancements & new features ===
- 1641 (add delayed calls feature) - 1563 (conversion to py3k and some other changes)
That's a lot of patches. My immediate concern is that test_asynchat is very flaky and fails often. Sometimes it causes other tests to fail. Is there a patch that addresses this? If you need examples, just look through the buildbot mails that mention test_asynchat in: http://mail.python.org/pipermail/python-checkins/
No, it's one patch. All of the fixes were performed mostly by myself last spring, tested and verified in personal servers, then re-used by Giampaolo in his async ftp server (which pointed out a few small bugs, which have been fixed).
Some platforms have more problems than others, but almost all platforms have failed test_asynchat at one point or another.
Certainly that is the case. And according to my reading of a few buildbot failures, aside from someone breaking asyncore itself, the failures seem to stem from the test being unable to create a port to listen on in order to test the server/client functionality. This is a buildbot configuration issue (per host), not an asyncore issue.
That was the case a long time (~3? months) ago, but hasn't been the case recently. See my recent message about the release.
I'll look for it tomorrow. For reference, searches of 'site:mail.python.org test_asynchat failure buildbot' only seem to produce the socket listen error. If there is a better incantation to get google to produce the proper errors (and/or a link), I would appreciate the help.
Please break up the patches into 2 sets and prioritize the patches with the set.
Set #1: Patches that have a test and doc updates if required Set #2: Patches that don't have a test or doc updates as required
For the patches that fall into Set #1, list them in priority order. Top priority would be a problem that fixes failures seen in the buildbots. Next priority would go to the patches that solve more serious problems. Post the results here. I will try to look at them.
For every patch you list, make sure that it conforms to the proper style (e.g, PEP 8) and is essentially perfect and ready for inclusion. This means that there is a single file to download that contains all the modifications. The changes are appropriately commented, lines are less than 80 characters, etc. One of the modifications should be an entry in Misc/NEWS.
I lied earlier; really there are two patches. The first is a patch to asyncore.py and asynchat.py . It addresses those bugs that Giampaolo has listed, it is tested, and works. The second patch is to update the documentation to mention the sample methods in asynchat for use as examples, as well as any other changes to the language used in the documentation that I had made last spring, but which are out of date from my posting of the original patch. I can update the documentation in the next week.
Can you provide a link to the patches? Do the patches include changes to test_asyncore and test_asynchat? The next release is April 2. I would like to commit any patches before Monday to ensure they are stable. Can you get me the patches by this Saturday?
See http://bugs.python.org/issue1736190 for an updated patch for the modules. The current test cases pass without issue, though we may want to add tests, which I need to look at the original patch and the original file from which it was created against, then compare it with the most recent changes to the tests from Facundo last June or July.
I should have the time to get patches for tests and documentation by Monday.
- Josiah
On Sun, Mar 30, 2008 at 7:44 PM, Josiah Carlson
I haven't really had time to update the tests/documentation, but again, I wasn't experiencing any strange test failures with asyncore/asynchat, nor have I been able to find the buildbot failures that you are referring to. Could someone please link the failures that are not related to being unable to discover a port number?
3 different platforms, 3 different problems: http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/27... http://www.python.org/dev/buildbot/all/x86%20FreeBSD%203%203.0/builds/0/step... http://www.python.org/dev/buildbot/all/x86%20XP-4%203.0/builds/643
According to the release schedule, we should have at least a couple more months for documentation and tests to be updated (I can get patches ready for alpha 3).
When you get the patches with tests and doc, I'll be happy to check in. n
On Mon, Mar 31, 2008 at 12:11 AM, Neal Norwitz
On Sun, Mar 30, 2008 at 7:44 PM, Josiah Carlson
wrote: I haven't really had time to update the tests/documentation, but again, I wasn't experiencing any strange test failures with asyncore/asynchat, nor have I been able to find the buildbot failures that you are referring to. Could someone please link the failures that are not related to being unable to discover a port number?
3 different platforms, 3 different problems:
http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/27... http://www.python.org/dev/buildbot/all/x86%20FreeBSD%203%203.0/builds/0/step... http://www.python.org/dev/buildbot/all/x86%20XP-4%203.0/builds/643
According to the release schedule, we should have at least a couple more months for documentation and tests to be updated (I can get patches ready for alpha 3).
When you get the patches with tests and doc, I'll be happy to check in.
n
I have updated the documentation, and as much of the tests as was required to pass on my Windows XP machine. In looking at the buildbots, I'm not seeing any more common issues. Unfortunately, I also hit the urls provided above too late, and I wasn't able to see the actual issues (if they still persist). If possible, the syntax in the documentation that I added should be checked, as I needed to convert from my older TeX docs to the new ReST doc format. I have attached the patch file, and am in the process of regaining access to the bug tracker. In the mean time, Giampaolo will be posting the patch to the tracker in issue 1736190. If anyone has any questions, please ask. - Josiah
participants (2)
-
Josiah Carlson
-
Neal Norwitz