asyncore fixes in Python 2.6 broke Zope's version of medusa
I am working on getting Zope to run (or at least, start) with Python 2.6. It actually starts right now after applying some patches, which is amazing on itself, but it dies right away due to changes in asyncore that break Zope's internal version of medusa. I've opened a bug against Zope on Launchpad, but someone suggested that it might actually be a bug in python, in the sense that it changed asyncore in a backwards-incompatible way. I wouldn't go that far, since I think it's more likely that Zope's version of medusa is poking into asyncore internals instead: https://bugs.edge.launchpad.net/zope2/+bug/280020 I suspect a patch similar to this might be needed: http://codereview.appspot.com/744/diff/1/23 Anyone cares to comment? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
Sidnei da Silva <sidnei@enfoldsystems.com> wrote:
I think there are real issues here with both asynchat and Medusa. Asynchat has been heavily re-written, and the "ac_out_buffer" has apparently disappeared. But "ac_out_buffer_size" is still there. That strikes me as odd, and probably means that asynchat.py needs more changes. However, Medusa (basically just an application layer on top of asyncore/asynchat) also needs to be re-written to take account of the changes in asynchat. Bill
On Wed, Oct 8, 2008 at 10:30 AM, Bill Janssen <janssen@parc.com> wrote:
Sidnei da Silva <sidnei@enfoldsystems.com> wrote:
I think there are real issues here with both asynchat and Medusa. Asynchat has been heavily re-written, and the "ac_out_buffer" has apparently disappeared. But "ac_out_buffer_size" is still there. That strikes me as odd, and probably means that asynchat.py needs more changes. However, Medusa (basically just an application layer on top of asyncore/asynchat) also needs to be re-written to take account of the changes in asynchat.
ac_out_buffer was removed because it is unneeded (we have a deque; why rely on an extra attribute?). ac_out_buffer_size still remains as a blocksize in which to pre-split outgoing data (if you have a 100k string you want to send, repeatedly slicing it as you are able to send pieces is slow, but pre-slicing chunks is fast, and generally results in being able to pass full chunk to the underlying TCP/IP stack). But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now. - Josiah
On Wed, Oct 8, 2008 at 6:26 PM, Josiah Carlson <josiah.carlson@gmail.com> wrote:
ac_out_buffer was removed because it is unneeded (we have a deque; why rely on an extra attribute?). ac_out_buffer_size still remains as a blocksize in which to pre-split outgoing data (if you have a 100k string you want to send, repeatedly slicing it as you are able to send pieces is slow, but pre-slicing chunks is fast, and generally results in being able to pass full chunk to the underlying TCP/IP stack).
But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now.
Great to know. So medusa should be changed similarly to the changes made to asynchat? Your suggestion on the bugtracker is even better: to subclass asynchat as much as possible. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
Sidnei da Silva <sidnei@enfoldsystems.com> wrote:
Great to know. So medusa should be changed similarly to the changes made to asynchat?
Hmmm. Been a long time since a Medusa release, but I need a working Medusa, too.
Your suggestion on the bugtracker is even better: to subclass asynchat as much as possible.
That's basically what Medusa is, a set of subclasses. Bill
Josiah Carlson <josiah.carlson@gmail.com> wrote:
But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now.
Just for completeness, I built a fresh 2.6, installed Medusa (from http://www.amk.ca/python/code/medusa.html), and it runs just fine (well, as well as it does on 2.5, anyway). I think this is a Zope issue. Bill
On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <janssen@parc.com> wrote:
Josiah Carlson <josiah.carlson@gmail.com> wrote:
But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now.
Just for completeness, I built a fresh 2.6, installed Medusa (from http://www.amk.ca/python/code/medusa.html), and it runs just fine (well, as well as it does on 2.5, anyway). I think this is a Zope issue.
Way back in the day, Zope monkeypatched whole parts of asyncore, replacing them with some of its own code. If that's still the case, this breakage should be no surprise. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote:
On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <janssen@parc.com> wrote:
Josiah Carlson <josiah.carlson@gmail.com> wrote:
But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now. Just for completeness, I built a fresh 2.6, installed Medusa (from http://www.amk.ca/python/code/medusa.html), and it runs just fine (well, as well as it does on 2.5, anyway). I think this is a Zope issue.
Way back in the day, Zope monkeypatched whole parts of asyncore, replacing them with some of its own code. If that's still the case, this breakage should be no surprise.
Zope has been hooking the 'asyncore.loop' function (wrapping it in order to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is the earliest checkout I have, and it was branched in early January 2002). Zope also began patched asyncore's logging functions in 2.7, and later updated that patch to make the logger use the stdlib 'logging' module. I think the change we need to make (in ZServer/medusa/http_server.py) is to emulate the new 'writeable' implementation in asynchat, at least when running under 2.6. Zope's 'ZServer.medusa.http_server.http_request.writable()' implementation is:: def writable (self): # this is just the normal async_chat 'writable', # here for comparison return self.ac_out_buffer or len(self.producer_fifo) The Python 2.5 asynchat.asynchat.writable does:: def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or # (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer == '') and self.producer_fifo.is_empty() and self.connected ) The Python 2.6 asynchat.asynchat.writable now does:: def writable (self): "predicate for inclusion in the writable for select()" return self.producer_fifo or (not self.connected) medusa 0.5.4's medusa.http_server.http_request doesn't override 'writable', but it does have a broken 'writeable_for_proxy': def writable_for_proxy (self): # this version of writable supports the idea of a 'stalled' # producer # [i.e., it's not ready to produce any output yet] This is # needed by # the proxy, which will be waiting for the magic combination of # 1) hostname resolved # 2) connection made # 3) data available. if self.ac_out_buffer: return 1 elif len(self.producer_fifo): p = self.producer_fifo.first() if hasattr (p, 'stalled'): return not p.stalled() else: return 1 Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI7ix3+gerLs4ltQ4RAldnAKC/QLJHmdE9dxInkWuIGja0gtSXYwCcCJcH 6XooEwW/AkJ1ntmGyxi8urM= =1kps -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
Guido van Rossum wrote:
Josiah Carlson <josiah.carlson@gmail.com> wrote:
But yes, zope needs to be changed to reflect the updated asyncore/asynchat semantics. Trust me; it's faster, cleaner, and easier to use now. Just for completeness, I built a fresh 2.6, installed Medusa (from http://www.amk.ca/python/code/medusa.html), and it runs just fine (well, as well as it does on 2.5, anyway). I think this is a Zope issue. Way back in the day, Zope monkeypatched whole parts of asyncore, replacing them with some of its own code. If that's still the case,
On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <janssen@parc.com> wrote: this breakage should be no surprise.
Zope has been hooking the 'asyncore.loop' function (wrapping it in order to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is the earliest checkout I have, and it was branched in early January 2002).
Zope also began patched asyncore's logging functions in 2.7, and later updated that patch to make the logger use the stdlib 'logging' module.
I think the change we need to make (in ZServer/medusa/http_server.py) is to emulate the new 'writeable' implementation in asynchat, at least when running under 2.6.
Zope's 'ZServer.medusa.http_server.http_request.writable()' implementation is::
def writable (self): # this is just the normal async_chat 'writable', # here for comparison return self.ac_out_buffer or len(self.producer_fifo)
The Python 2.5 asynchat.asynchat.writable does::
def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or # (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer == '') and self.producer_fifo.is_empty() and self.connected )
The Python 2.6 asynchat.asynchat.writable now does::
def writable (self): "predicate for inclusion in the writable for select()" return self.producer_fifo or (not self.connected)
medusa 0.5.4's medusa.http_server.http_request doesn't override 'writable', but it does have a broken 'writeable_for_proxy':
def writable_for_proxy (self): # this version of writable supports the idea of a 'stalled' # producer # [i.e., it's not ready to produce any output yet] This is # needed by # the proxy, which will be waiting for the magic combination of # 1) hostname resolved # 2) connection made # 3) data available. if self.ac_out_buffer: return 1 elif len(self.producer_fifo): p = self.producer_fifo.first() if hasattr (p, 'stalled'): return not p.stalled() else: return 1
*crickets*. The 2.6 version of asyncore *also* breaks supervisord, which does *not* use Zope's medusa: it uses medusa 0.5.4. How is anybody supposed to write a package which sits atop a library like asyncore in a fashion portable across Python versions? The changes to the implementation in 2.6 (there is no real API) can't be reconciled, AFAICT. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJrZG0+gerLs4ltQ4RAnsmAJ9v/vPkHgE3AdP5ngVuYaKlxDGhJACgsCi2 3awbUffi2BU41qQgd6eJV18= =WBt6 -----END PGP SIGNATURE-----
On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver <tseaver@palladion.com> wrote:
How is anybody supposed to write a package which sits atop a library like asyncore in a fashion portable across Python versions? The changes to the implementation in 2.6 (there is no real API) can't be reconciled, AFAICT.
This seems to be the crux of the problem with asyncore, ever since it was added to the stdlib -- there's no real API, so every change potentially breaks something. I wish we could start over with a proper design under a new name. Maybe packages requiring asyncore functionality should just copy the version of asyncore they like into their own three and stick with that. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 08:46 pm, guido@python.org wrote:
This seems to be the crux of the problem with asyncore, ever since it was added to the stdlib -- there's no real API, so every change potentially breaks something. I wish we could start over with a proper design under a new name.
Might I suggest "reactor"... or possibly "twisted", as that new name? ;-) (Sorry, I was trying to avoid this thread, but that was an opening I could drive a truck through). In all seriousness, I seem to recall that Thomas Wouters was interested in doing integrating some portion of Twisted core into the standard library as of last PyCon. I mention him specifically by name in the hopes that it will jog his memory. At the very least, this might serve as a basis for an abstract API for asyncore: http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IPr...
On Tue, Mar 3, 2009 at 1:17 PM, <glyph@divmod.com> wrote:
On 08:46 pm, guido@python.org wrote:
This seems to be the crux of the problem with asyncore, ever since it was added to the stdlib -- there's no real API, so every change potentially breaks something. I wish we could start over with a proper design under a new name.
Might I suggest "reactor"... or possibly "twisted", as that new name? ;-)
(Sorry, I was trying to avoid this thread, but that was an opening I could drive a truck through).
In all seriousness, I seem to recall that Thomas Wouters was interested in doing integrating some portion of Twisted core into the standard library as of last PyCon. I mention him specifically by name in the hopes that it will jog his memory.
At the very least, this might serve as a basis for an abstract API for asyncore:
http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IPr...
I hope we have learned from asyncore that a future-proof and maximally flexible design of such an API is not easy. This is one of those cases where I believe thinking hard up front will really pay off. In this case I would recommend going through a full-blown PEP process before committing to an API, to avoid making the same mistake twice. (I don't mean this as a dig at Twisted, just as an observation about this particular design problem. If Twisted solves all the problems already, writing the PEP should be easy.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 3 Mar, 10:20 pm, guido@python.org wrote:
On Tue, Mar 3, 2009 at 1:17 PM, <glyph@divmod.com> wrote:
At the very least, this might serve as a basis for an abstract API for asyncore:
http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IPr...
I hope we have learned from asyncore that a future-proof and maximally flexible design of such an API is not easy. This is one of those cases where I believe thinking hard up front will really pay off. In this case I would recommend going through a full-blown PEP process before committing to an API, to avoid making the same mistake twice. (I don't mean this as a dig at Twisted, just as an observation about this particular design problem. If Twisted solves all the problems already, writing the PEP should be easy.)
I didn't see it as a dig at Twisted; you're right, this is a hard problem. Twisted doesn't solve all the problems perfectly, either; much has been made of some rather unfortunate nuances of the IProducer and IConsumer interfaces. I also regret that no Twisted people (myself included) seem to have the time to go through the PEP process and get something accepted. Even if Twisted's interfaces aren't perfect, there's a lot of working application code which demonstrates that they are sufficient. Perhaps more importantly, they are future-proof. Every proposal we've come up with for addressing the more subtle shortcomings of these APIs has been completely backwards-compatible; there are no APIs with vague or broken semantics, just sub-optimal spellings. The fact that there are already multiple reactor implementations which share very little code helps. If someone who has PEP-writing skills and some free time is updating asyncore to be more modern, I'd be very happy to walk that person through Twisted's API design and explain how either portions of our implementation could be lifted for the stdlib, or how a "high level" layer could be written into asyncore so that the author of a particular chunk of low-level networking code could ignore whether they're using the stdlib mainloop or one of the Twisted ones.
On Tue, Mar 3, 2009 at 9:31 PM, <glyph@divmod.com> wrote:
On 3 Mar, 10:20 pm, guido@python.org wrote:
On Tue, Mar 3, 2009 at 1:17 PM, <glyph@divmod.com> wrote:
At the very least, this might serve as a basis for an abstract API for asyncore:
http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IPr...
I hope we have learned from asyncore that a future-proof and maximally flexible design of such an API is not easy. This is one of those cases where I believe thinking hard up front will really pay off. In this case I would recommend going through a full-blown PEP process before committing to an API, to avoid making the same mistake twice. (I don't mean this as a dig at Twisted, just as an observation about this particular design problem. If Twisted solves all the problems already, writing the PEP should be easy.)
I didn't see it as a dig at Twisted; you're right, this is a hard problem. Twisted doesn't solve all the problems perfectly, either; much has been made of some rather unfortunate nuances of the IProducer and IConsumer interfaces.
I also regret that no Twisted people (myself included) seem to have the time to go through the PEP process and get something accepted.
Same here.
Even if Twisted's interfaces aren't perfect, there's a lot of working application code which demonstrates that they are sufficient. Perhaps more importantly, they are future-proof. Every proposal we've come up with for addressing the more subtle shortcomings of these APIs has been completely backwards-compatible; there are no APIs with vague or broken semantics, just sub-optimal spellings. The fact that there are already multiple reactor implementations which share very little code helps.
Sounds like it's not so much the code that's future proof but the process used for evolving it. That seems to be missing for asyncore. :-(
If someone who has PEP-writing skills and some free time is updating asyncore to be more modern, I'd be very happy to walk that person through Twisted's API design and explain how either portions of our implementation could be lifted for the stdlib, or how a "high level" layer could be written into asyncore so that the author of a particular chunk of low-level networking code could ignore whether they're using the stdlib mainloop or one of the Twisted ones.
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido@python.org> wrote:
Sounds like it's not so much the code that's future proof but the process used for evolving it. That seems to be missing for asyncore. :-(
Turning the issue around a bit, has anyone considered polishing up the current fix to restore it's backwards compatibility, instead of starting a discussion about a full-blown replacement? I think that would be enough for most asyncore users (or even the couple few affected) for the moment, and then we can think about a possible future replacement. -- Sidnei da Silva Canonical Ltd. Landscape · Changing the way you manage your systems http://landscape.canonical.com
On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva <sidnei.da.silva@gmail.com> wrote:
On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido@python.org> wrote:
Sounds like it's not so much the code that's future proof but the process used for evolving it. That seems to be missing for asyncore. :-(
Turning the issue around a bit, has anyone considered polishing up the current fix to restore it's backwards compatibility, instead of starting a discussion about a full-blown replacement?
I think that would be enough for most asyncore users (or even the couple few affected) for the moment, and then we can think about a possible future replacement.
If it can be done while maintaining backwards compatibility with both the 2.6 version and the pre-2.6 version, that would be great of course! But can it? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva <sidnei.da.silva@gmail.com> wrote:
On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido@python.org> wrote:
Sounds like it's not so much the code that's future proof but the process used for evolving it. That seems to be missing for asyncore. :-(
Turning the issue around a bit, has anyone considered polishing up the current fix to restore it's backwards compatibility, instead of starting a discussion about a full-blown replacement?
I think that would be enough for most asyncore users (or even the couple few affected) for the moment, and then we can think about a possible future replacement.
If it can be done while maintaining backwards compatibility with both the 2.6 version and the pre-2.6 version, that would be great of course! But can it?
Is it really necessary to retain compatibility with the Python 2.6 version? Python 2.6.0 and Python 2.6.1 contain a regression (as compared to basically all previous versions of Python) which prevents asyncore-based programs which are years old from working on them. Restoring the pre-2.6 behavior will fix these old, presumably stable, widely used programs for users who install 2.6.2 and newer. The downside (which you were imagining, I'm sure) is that any new software developed against the Python 2.6.0 or 2.6.1 behavior will then break in 2.6.2 and later. While this is unfortunate, it is clearly the far lesser of two evils. The choice must be made, though. Either leave old software broken or break new software. Just because the "leave old software broken" choice is made through inaction doesn't make it the better choice (though obviously since it requires action, someone will have to do it, and I'm not volunteering - if inaction is the choice because no one wants to do the work, fine, but that's a different motivation than avoiding breaking newly written software). So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior. Jean-Paul
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
- Show quoted text - On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva <sidnei.da.silva@gmail.com> wrote:
On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido@python.org> wrote:
Sounds like it's not so much the code that's future proof but the process used for evolving it. That seems to be missing for asyncore. :-(
Turning the issue around a bit, has anyone considered polishing up the current fix to restore it's backwards compatibility, instead of starting a discussion about a full-blown replacement?
I think that would be enough for most asyncore users (or even the couple few affected) for the moment, and then we can think about a possible future replacement.
If it can be done while maintaining backwards compatibility with both the 2.6 version and the pre-2.6 version, that would be great of course! But can it?
Is it really necessary to retain compatibility with the Python 2.6 version? Python 2.6.0 and Python 2.6.1 contain a regression (as compared to basically all previous versions of Python) which prevents asyncore-based programs which are years old from working on them. Restoring the pre-2.6 behavior will fix these old, presumably stable, widely used programs for users who install 2.6.2 and newer.
The downside (which you were imagining, I'm sure) is that any new software developed against the Python 2.6.0 or 2.6.1 behavior will then break in 2.6.2 and later. While this is unfortunate, it is clearly the far lesser of two evils. The choice must be made, though. Either leave old software broken or break new software. Just because the "leave old software broken" choice is made through inaction doesn't make it the better choice (though obviously since it requires action, someone will have to do it, and I'm not volunteering - if inaction is the choice because no one wants to do the work, fine, but that's a different motivation than avoiding breaking newly written software).
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g. import asyncore asyncore.python_25_compat = True Note that this "API" is designed to work in 2.5 as well. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote: [snip]
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g.
import asyncore asyncore.python_25_compat = True
Note that this "API" is designed to work in 2.5 as well. :-)
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)? Jean-Paul
On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote: [snip]
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g.
import asyncore asyncore.python_25_compat = True
Note that this "API" is designed to work in 2.5 as well. :-)
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)?
The same as always. We don't change APIs in bugfix releases. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, 4 Mar 2009 10:54:19 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote: [snip]
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g.
import asyncore asyncore.python_25_compat = True
Note that this "API" is designed to work in 2.5 as well. :-)
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)?
The same as always. We don't change APIs in bugfix releases.
Okay. Thanks for explaining. Jean-Paul
Guido van Rossum wrote:
On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote: [snip]
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior. Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g.
import asyncore asyncore.python_25_compat = True
Note that this "API" is designed to work in 2.5 as well. :-)
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)?
The same as always. We don't change APIs in bugfix releases.
Which unfortunately stops us from categorizing API breakages as bugs. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote: [snip]
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior. It does less harm than leaving the current behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well. If that makes it impossible to also maintain the 2.5 behavior, perhaps some flag could be added to restore 2.5 compatibility, e.g.
import asyncore asyncore.python_25_compat = True
Note that this "API" is designed to work in 2.5 as well. :-)
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)?
The same as always. We don't change APIs in bugfix releases.
OK, seems reasonable. But in this case, isn't the broken API the bug that's being fixed? Do we need a different way to fix broken APIs in bugfix releases? Bill
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido@python.org> wrote:
The same as always. We don't change APIs in bugfix releases.
This question is actually for the Zope folks and others who have had problems with the 2.6 asyncore/asynchat: Are any of the problems due to a change in the documented API... or are they all due to changes in undocumented internals that your code relied on? Myself, the only change that I noticed going from 2.5 to 2.6 was that the undocumented signature of asynchat.__init__() changed from taking a parameter called "conn" to one with the same meaning called "sock". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
Daniel Stutzbach wrote:
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido@python.org> wrote:
The same as always. We don't change APIs in bugfix releases.
This question is actually for the Zope folks and others who have had problems with the 2.6 asyncore/asynchat:
Are any of the problems due to a change in the documented API... or are they all due to changes in undocumented internals that your code relied on?
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules. Medusa itself (from which asyncore and asynchat were derived) appears to itself break with the changes to asyncore/asynchat in 2.6 (at least it appears to use attributes like "ac_out_buffer" which were removed in 2.6; this is not "Zope's version"; there is no such animal; this is plain old Medusa 0.5.4). Count me in as one who believes that it would be the lesser of two evils to revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2 rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its asyncore/asynchat more forward compatible with 3.0's than backwards compatible with 2.5's and prior; this was just a mistake and keeping old code running should trump any theoretical or puritanical benefit in a dot release. - C
On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough <chrism@plope.com> wrote:
Daniel Stutzbach wrote:
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido@python.org> wrote:
The same as always. We don't change APIs in bugfix releases.
This question is actually for the Zope folks and others who have had problems with the 2.6 asyncore/asynchat:
Are any of the problems due to a change in the documented API... or are they all due to changes in undocumented internals that your code relied on?
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules. Medusa itself (from which asyncore and asynchat were derived) appears to itself break with the changes to asyncore/asynchat in 2.6 (at least it appears to use attributes like "ac_out_buffer" which were removed in 2.6; this is not "Zope's version"; there is no such animal; this is plain old Medusa 0.5.4).
Count me in as one who believes that it would be the lesser of two evils to revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2 rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its asyncore/asynchat more forward compatible with 3.0's than backwards compatible with 2.5's and prior; this was just a mistake and keeping old code running should trump any theoretical or puritanical benefit in a dot release.
Then I'd like to hear from the folks who made and reviewed those changes to asyncore in 2.6. I can't imagine the changes were done without good intentions. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, Mar 4, 2009 at 1:31 PM, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough <chrism@plope.com> wrote:
Daniel Stutzbach wrote:
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido@python.org> wrote:
The same as always. We don't change APIs in bugfix releases.
This question is actually for the Zope folks and others who have had problems with the 2.6 asyncore/asynchat:
Are any of the problems due to a change in the documented API... or are they all due to changes in undocumented internals that your code relied on?
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules. Medusa itself (from which asyncore and asynchat were derived) appears to itself break with the changes to asyncore/asynchat in 2.6 (at least it appears to use attributes like "ac_out_buffer" which were removed in 2.6; this is not "Zope's version"; there is no such animal; this is plain old Medusa 0.5.4).
Count me in as one who believes that it would be the lesser of two evils to revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2 rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its asyncore/asynchat more forward compatible with 3.0's than backwards compatible with 2.5's and prior; this was just a mistake and keeping old code running should trump any theoretical or puritanical benefit in a dot release.
Then I'd like to hear from the folks who made and reviewed those changes to asyncore in 2.6. I can't imagine the changes were done without good intentions.
Interestingly enough, I just accidentally found this thread with an unrelated web search, but it makes sense to reply. Why the changes to asyncore: 1) the internals of asyncore were inefficient 2) the API and general behavior of the handle_*(), readable() and writable() methods are unchanged, which are *exactly* what a user should be overriding with subclassing 3) anyone modifying the *internals* of the implementation for the sake of speed, shouldn't need to anymore Maybe I have misread the posts in this thread, but does anyone have any examples of their code breaking when only overriding handle_*(), readable(), and writable() methods? - Josiah
Chris McDonough <chrism@plope.com> wrote:
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules.
What I would like to see is a module that provides a low-level API for doing cross-platform asynchronous IO. The two necessary parts are: * a wrapper that allows non-blocking reads and writes on channels (sockets, file descriptors, serial ports, etc) * a select() or epoll like interface that allows waiting on multiple channels The implementation requires some intricate and platform specific code which is why it would be nice to be a standard library feature. I'm sure that Twisted has the necessary parts but the problem, IMHO, is that it does so much more else. Neil
On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer <nas@arctrix.com> wrote:
What I would like to see is a module that provides a low-level API for doing cross-platform asynchronous IO. The two necessary parts are:
* a wrapper that allows non-blocking reads and writes on channels (sockets, file descriptors, serial ports, etc)
* a select() or epoll like interface that allows waiting on multiple channels
Two thoughts: If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary. On Windows, sockets, pipes, and files are each completely distinct types with their own functions and unifying them under one select()-like interface requires faking it using threads behind-the-scenes AFAIK. Personally, I'd be happy with continuing to only support socket objects on Windows. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
Daniel Stutzbach wrote:
If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary.
No, but there should be an interface that lets you say "when something comes in on this fd, call this function for me". In other words it should be a light wrapper around select/poll/whatever that provides a callback interface. -- Greg
On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary.
No, but there should be an interface that lets you say "when something comes in on this fd, call this function for me".
In other words it should be a light wrapper around select/poll/whatever that provides a callback interface.
A read callback, a write callback. What about close, error, connect, and accept callbacks? I hate to say it (not really), but that's pretty much the handle_*() methods of asyncore :/ . - Josiah
Josiah Carlson <josiah.carlson@gmail.com> wrote:
On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary.
No, but there should be an interface that lets you say "when something comes in on this fd, call this function for me".
In other words it should be a light wrapper around select/poll/whatever that provides a callback interface.
A read callback, a write callback. What about close, error, connect, and accept callbacks?
I hate to say it (not really), but that's pretty much the handle_*() methods of asyncore :/ .
What asyncore was missing was a timer API (a way to register functions to be called periodically). Then it would be pretty much like any other event loop system. Bill
On Thu, Mar 5, 2009 at 1:09 PM, Bill Janssen <janssen@parc.com> wrote:
Josiah Carlson <josiah.carlson@gmail.com> wrote:
On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary.
No, but there should be an interface that lets you say "when something comes in on this fd, call this function for me".
In other words it should be a light wrapper around select/poll/whatever that provides a callback interface.
A read callback, a write callback. What about close, error, connect, and accept callbacks?
I hate to say it (not really), but that's pretty much the handle_*() methods of asyncore :/ .
What asyncore was missing was a timer API (a way to register functions to be called periodically). Then it would be pretty much like any other event loop system.
There are two variants of patches to offer timer API functionality in the bug tracker right now. One from Giampaolo that uses a variant of Twisted's scheduler, one from me that uses an updated sched.py . Giampaolo's is more complete (it has documentation and tests), but mine is more efficient with nontrivial task lists. - Josiah
Josiah Carlson wrote:
A read callback, a write callback. What about close, error, connect, and accept callbacks?
Yep, all those as well.
I hate to say it (not really), but that's pretty much the handle_*() methods of asyncore :/ .
Well, then, what's the problem? Is there anything else people want that asyncore doesn't provide, or is it just a matter of nailing down the existing API a little? -- Greg
Neil Schemenauer wrote:
What I would like to see is a module that provides a low-level API for doing cross-platform asynchronous IO. The two necessary parts are:
* a wrapper that allows non-blocking reads and writes on channels (sockets, file descriptors, serial ports, etc)
* a select() or epoll like interface that allows waiting on multiple channels
+1 -- Greg
Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
Neil Schemenauer wrote:
What I would like to see is a module that provides a low-level API for doing cross-platform asynchronous IO. The two necessary parts are:
* a wrapper that allows non-blocking reads and writes on channels (sockets, file descriptors, serial ports, etc)
For starters, since py3k is supposed to support non-blocking IO, why not write a portable API to make a raw file or socket IO object non-blocking? (I'm only suggesting it here, I don't intend to do this myself)
Antoine Pitrou wrote:
For starters, since py3k is supposed to support non-blocking IO, why not write a portable API to make a raw file or socket IO object non-blocking?
I think we need to be clearer what we mean when we talk about non-blocking in this context. Normally when you're using select/poll you *don't* make the underlying file descriptor non-blocking in the OS sense. The non-blockingness comes from the fact that you're using select/poll to make sure the fd is ready before you access it. So I don't think it makes sense to talk about having a non-blocking API as a separate thing from a select/poll wrapper. The select/poll wrapper *is* the non-blocking API. -- Greg
On 07:30 pm, nas@arctrix.com wrote:
Chris McDonough <chrism@plope.com> wrote:
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules.
The implementation requires some intricate and platform specific code which is why it would be nice to be a standard library feature.
I'm sure that Twisted has the necessary parts but the problem, IMHO, is that it does so much more else.
... which is exactly why I have volunteered to explain to someone how to separate the core event-loop bits (suitable for inclusion in the standard library) from the huge pile of protocol implementations which are not necessarily useful. Despite the FUD to the contrary, Twisted's internal factoring is quite good; it's not a single, undifferentiated pile of code. Plus, at this point, we're not even talking about actually putting any Twisted code into the standard library, just standardizing the "protocol" API, which basically boils down to: connectionMade() -> your connection has begun dataReceived(data) -> you got some bytes, handle them connectionLost(reason) -> your connection has gone away (with an object explaining why) and the inverse, "transport", which is: write(data) -> deliver some data to the dataReceived on the other end of this connection (non-blocking, with buffering) loseConnection() -> goodbye There are a few other minor details related to how you set these up to talk to each other and tell when the out-buffer is empty, but it's all pretty straightforward. The main point is that you don't ever call recv() or send() and deal with buffering or handling weird errno values. For example, if your connection goes away, the notification you get is "your connection went away", not "oops you tried to read some bytes, but your connection was gone by the time you tried, even though I just told you it was ready for reading" or other similarly obtuse failure modes.
On Thu, Mar 5, 2009 at 3:11 PM, <glyph@divmod.com> wrote:
On 07:30 pm, nas@arctrix.com wrote:
Chris McDonough <chrism@plope.com> wrote:
As far as I can tell, asyncore/asynchat is all "undocumented internals". Any use of asyncore in anger will use internals; there never was any well-understood API to these modules.
The implementation requires some intricate and platform specific code which is why it would be nice to be a standard library feature.
I'm sure that Twisted has the necessary parts but the problem, IMHO, is that it does so much more else.
... which is exactly why I have volunteered to explain to someone how to separate the core event-loop bits (suitable for inclusion in the standard library) from the huge pile of protocol implementations which are not necessarily useful.
Despite the FUD to the contrary, Twisted's internal factoring is quite good; it's not a single, undifferentiated pile of code. Plus, at this point, we're not even talking about actually putting any Twisted code into the standard library, just standardizing the "protocol" API, which basically boils down to:
connectionMade() -> your connection has begun dataReceived(data) -> you got some bytes, handle them connectionLost(reason) -> your connection has gone away (with an object explaining why)
and the inverse, "transport", which is:
write(data) -> deliver some data to the dataReceived on the other end of this connection (non-blocking, with buffering) loseConnection() -> goodbye
There are a few other minor details related to how you set these up to talk to each other and tell when the out-buffer is empty, but it's all pretty straightforward. The main point is that you don't ever call recv() or send() and deal with buffering or handling weird errno values. For example, if your connection goes away, the notification you get is "your connection went away", not "oops you tried to read some bytes, but your connection was gone by the time you tried, even though I just told you it was ready for reading" or other similarly obtuse failure modes.
Those "obtuse failure modes" are handled fairly well by asynchat. It works pretty well, and includes sample implementations to get you 99% of the way towards the 5-method API you describe (which is why a lot of people use asynchat instead of asyncore). If it weren't for the fact that asynchat had a previously existing API (including the .push(), collect_incoming_data(), and found_terminator()), I think it could stand for methods to make it fit better with asyncore's handle_*() methods. - Josiah
glyph@divmod.com <glyph@divmod.com> wrote:
... which is exactly why I have volunteered to explain to someone how to separate the core event-loop bits (suitable for inclusion in the standard library) from the huge pile of protocol implementations which are not necessarily useful.
Despite the FUD to the contrary, Twisted's internal factoring is quite good; it's not a single, undifferentiated pile of code. Plus, at this point, we're not even talking about actually putting any Twisted code into the standard library, just standardizing the "protocol" API, which basically boils down to: [...]
This sounds great. I'm interested on working on this since it scratches an itch of mine but I don't know if I will have time. Do you think if this part of Twisted became part of the standard library that it would be used by Twisted or would it continue to have its own version? Neil
Glyph> ... which is exactly why I have volunteered to explain to someone Glyph> how to separate the core event-loop bits (suitable for inclusion Glyph> in the standard library) from the huge pile of protocol Glyph> implementations which are not necessarily useful. Neil> This sounds great. I'm interested on working on this since it Neil> scratches an itch of mine but I don't know if I will have time. Neil> Do you think if this part of Twisted became part of the standard Neil> library that it would be used by Twisted or would it continue to Neil> have its own version? Anybody interested in working on this at a PyCon Sprint? I won't be attending the conference proper, but plan to spend a couple days sprinting, one on Mailman/SpamBayes integration and one on Python core stuff. This might fit well into my Python core "slot". I will probably have little time before the sprint to do much, but any brain dumps or Twisted pointers people could give me in the interim would be appreciated. Skip
On 02:28 am, skip@pobox.com wrote:
Glyph> ... which is exactly why I have volunteered to explain to someone Glyph> how to separate the core event-loop bits
Neil> This sounds great.
Anybody interested in working on this at a PyCon Sprint? I won't be attending the conference proper, but plan to spend a couple days sprinting, one on Mailman/SpamBayes integration and one on Python core stuff. This might fit well into my Python core "slot". I will probably have little time before the sprint to do much, but any brain dumps or Twisted pointers people could give me in the interim would be appreciated.
I'll try to make sure we get those notes to you in advance of the sprints :). I should be at the Twisted sprint the whole time - will you be at the physical sprint, or following along at home?
Glyph> I'll try to make sure we get those notes to you in advance of the Glyph> sprints :). I should be at the Twisted sprint the whole time - Glyph> will you be at the physical sprint, or following along at home? Thanks. I will be at the physical sprints. (I live in the Chicago area.) Skip
On Sat, Mar 7, 2009 at 8:28 PM, <skip@pobox.com> wrote:
Anybody interested in working on this at a PyCon Sprint? I won't be attending the conference proper, but plan to spend a couple days sprinting,
I'll be there and interested. :) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On 02:46 pm, daniel@stutzbachenterprises.com wrote:
On Sat, Mar 7, 2009 at 8:28 PM, <skip@pobox.com> wrote:
Anybody interested in working on this at a PyCon Sprint? I won't be attending the conference proper, but plan to spend a couple days sprinting,
I'll be there and interested. :)
Great! I plan to hold both of you to that :).
On 01:50 am, nas@arctrix.com wrote:
glyph@divmod.com <glyph@divmod.com> wrote:
we're not even talking about actually putting any Twisted code into the standard library, just standardizing the "protocol" API, which basically boils down to: [...]
This sounds great. I'm interested on working on this since it scratches an itch of mine but I don't know if I will have time. Do you think if this part of Twisted became part of the standard library that it would be used by Twisted or would it continue to have its own version?
As I said above, right now I'm just talking about the interface. There are several implementations of the main loop that call those interfaces, as well as test implementations. Of course I hope we can eliminate some redundancy, but that's step 2: before we refactor, we have to make the code that we're refactoring actually repetitive rather than just similar.
On 06:51 pm, exarkun@divmod.com wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido@python.org> wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
So, as a disinterested party in this specific case, I'd say revert to the pre-2.6 behavior.
Sorry, but I really do think that we should maintain backward compatibility *within* the 2.6 series as well.
But why? The argument I made had the objective of minimizing developer effort. What's the objective of maintaining backward compatibility within the 2.6 series in this case (sorry if it appeared earlier in this thread and I missed it)?
Python's compatibility policy dictates that compatibility between micro- version releases be maintained *extremely* carefully. As you all well know, normally I'd be strongly in favor of adhering to such a policy. However, the ostensible purpose of the policy here is that users can *always* install micro-version upgrades to get bugfixes and security fixes, without worrying about compatibility upgrades. Maintaining compatibility with the 2.6.x version of asyncore presupposes that *someone* has written some software against that version of asyncore and it might break if they installed an upgrade, though. If that's the case - if there's even one person who has written or run any asyncore software against the version in 2.6 - then I think maintaining bug-for-bug compatibility is important. However, my guess - and I'm assuming that JP was thinking the same thing - is that literally nobody has done that, or even *would* ever do that, so there's no software anywhere in the world that would break. asyncore is relatively unpopular (thanks in part to the excellent alternatives :-)); its major users are the same people who have already complained. Maybe it's too late to do something like this for 2.6, or even 3.0, but I've thought on a few occasions that projects (both Python and Twisted, at least) could use a special very-low-traffic policy-deviations mailing list for "We *REALLY* don't think this is going to ever break anything anyone has actually done, but just to be sure..." notifications for situations like this. The implications being that (A) _everyone_ who uses the software should subscribe, and (B) if anyone ever responded to a message, the deviation wouldn't take place. Anyway, I'm also a disinterested observer, so please don't take this as a strong endorsement of doing the fix in 2.6; the set-an-attribute-to- fix-it idea is fine by me too. I thought I'd lay out the reasoning for violating an otherwise extremely reasonable policy, though.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 4, 2009, at 2:44 PM, glyph@divmod.com wrote:
Maintaining compatibility with the 2.6.x version of asyncore presupposes that *someone* has written some software against that version of asyncore and it might break if they installed an upgrade, though. If that's the case - if there's even one person who has written or run any asyncore software against the version in 2.6 - then I think maintaining bug-for-bug compatibility is important.
However, my guess - and I'm assuming that JP was thinking the same thing - is that literally nobody has done that, or even *would* ever do that, so there's no software anywhere in the world that would break. asyncore is relatively unpopular (thanks in part to the excellent alternatives :-)); its major users are the same people who have already complained.
FWIW, I use smtpd.py and a few of the asyncore APIs (.loop(), .socket_map.clear(), and .close_all()) in the Mailman 3 test suite. That only works on Python 2.6 and I don't recall even a hiccup when moving from 2.5 to 2.6. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSa7kiXEjvBPtnXfVAQI2BQP/XD71ipSv5OhtdOJt+OF2jbfOyFU/2vPR iJ33g9JJ2A0CvDWXmc81t9hMH3U8waagkXaDUF4r2MNM3JbONSjQGGZqX6X/1UOB XCT+jNOrbRw2VQR0qjrweQ5A8u0Y1zsDo/rD4CXe0K1pFFWLubaU3lVgqEBMBL8r xvf77EMoUuA= =vmM5 -----END PGP SIGNATURE-----
On 4 Mar, 08:28 pm, barry@python.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 4, 2009, at 2:44 PM, glyph@divmod.com wrote:
Maintaining compatibility with the 2.6.x version of asyncore presupposes that *someone* has written some software against that version of asyncore and it might break if they installed an upgrade, though.
FWIW, I use smtpd.py and a few of the asyncore APIs (.loop(), .socket_map.clear(), and .close_all()) in the Mailman 3 test suite. That only works on Python 2.6 and I don't recall even a hiccup when moving from 2.5 to 2.6.
Well, in that case, I withdraw even my disinterested suggestion :).
Glyph, On Mar 4, 2009, at 12:31 AM, glyph@divmod.com wrote:
If someone who has PEP-writing skills and some free time is updating asyncore to be more modern, I'd be very happy to walk that person through Twisted's API design and explain how either portions of our implementation could be lifted for the stdlib, or how a "high level" layer could be written into asyncore so that the author of a particular chunk of low-level networking code could ignore whether they're using the stdlib mainloop or one of the Twisted ones.
I spent about a half hour sometime in the last month talking this through with Itamar, though not in great detail. I'd be interested in sitting down with both of you and trying to establish more precisely how much work is necessary to get something to actually happen here. I won't outright promise a PEP, but I'll promise at the very least to write down elaborate notes that someone could turn into a PEP relatively straightforwardly. Deal? Cheers, -- Ivan Krstić <krstic@solarsail.hcs.harvard.edu> | http://radian.org
On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić < krstic@solarsail.hcs.harvard.edu> wrote:
I spent about a half hour sometime in the last month talking this through with Itamar, though not in great detail. I'd be interested in sitting down with both of you and trying to establish more precisely how much work is necessary to get something to actually happen here. I won't outright promise a PEP, but I'll promise at the very least to write down elaborate notes that someone could turn into a PEP relatively straightforwardly. Deal?
Will any or all of you be at PyCon? I'd be willing to put in the extra work to turn your notes into a PEP. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
Daniel Stutzbach wrote:
On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić <krstic@solarsail.hcs.harvard.edu <mailto:krstic@solarsail.hcs.harvard.edu>> wrote:
I spent about a half hour sometime in the last month talking this through with Itamar, though not in great detail. I'd be interested in sitting down with both of you and trying to establish more precisely how much work is necessary to get something to actually happen here. I won't outright promise a PEP, but I'll promise at the very least to write down elaborate notes that someone could turn into a PEP relatively straightforwardly. Deal?
Will any or all of you be at PyCon? I'd be willing to put in the extra work to turn your notes into a PEP.
OPEN SPACE! -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
On 4 Mar, 09:14 pm, krstic@solarsail.hcs.harvard.edu wrote:
I spent about a half hour sometime in the last month talking this through with Itamar, though not in great detail. I'd be interested in sitting down with both of you and trying to establish more precisely how much work is necessary to get something to actually happen here. I won't outright promise a PEP, but I'll promise at the very least to write down elaborate notes that someone could turn into a PEP relatively straightforwardly. Deal?
Absolutely. I really appreciate the offer. As the other gentlemen suggested, PyCon would be an ideal venue for doing this. Are you going to be there? I'll hopefully be seeing your talk this evening, but I suspect that would be a pretty bad venue to try to get this done ;-).
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote:
On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver <tseaver@palladion.com> wrote:
How is anybody supposed to write a package which sits atop a library like asyncore in a fashion portable across Python versions? The changes to the implementation in 2.6 (there is no real API) can't be reconciled, AFAICT.
This seems to be the crux of the problem with asyncore, ever since it was added to the stdlib -- there's no real API, so every change potentially breaks something. I wish we could start over with a proper design under a new name. Maybe packages requiring asyncore functionality should just copy the version of asyncore they like into their own three and stick with that.
That was the very solution Chris came up with earlier today: "stick a fork in it, its done!" Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJraty+gerLs4ltQ4RAjBSAJ4niecZJusKY4XiioJ18mdhdMixxQCfWvcQ Dwkh1ZBuxtGRbhUI4qy96Sc= =ms0I -----END PGP SIGNATURE-----
participants (17)
-
Antoine Pitrou -
Barry Warsaw -
Bill Janssen -
Chris McDonough -
Daniel Stutzbach -
glyph＠divmod.com -
Greg Ewing -
Guido van Rossum -
Ivan Krstić -
Jean-Paul Calderone -
Josiah Carlson -
Neil Schemenauer -
Sidnei da Silva -
Sidnei da Silva -
skip＠pobox.com -
Steve Holden -
Tres Seaver