Re: [Python-Dev] Trial balloon: microthreads library in stdlib
On 12 Feb, 05:11 pm, richard.m.tew@gmail.com wrote:
On 2/12/07, Tristan Seligmann <mithrandi-python-dev@mithrandi.za.net> wrote:
* Richard Tew <richard.m.tew@gmail.com> [2007-02-12 13:46:43 +0000]:
Perhaps there is a better way. And I of course have no concept of how this might be done on other platforms.
Building on an existing framework that does this seems better than reinventing the wheel, for something of this magnitude.
This to me seems to be the low level support you would build something like Twisted on top of. Pushing Twisted so that others can get it seems a little over the top.
It sounds like you don't really understand what Twisted is, what it does, or the difficulty involved in building that "low level" so that it's usable by something like Twisted. Tristan is correct: this should be a patch against Twisted, or perhaps as a separate library that could implement a reactor. I have no problem with other, competing event-driven mechanisms being developed; in fact, I think it's great that the style of programming is getting some attention. But this is not a robust and straightforward wrapping of some lower level. This is an entirely new, experimental project, and the place to start developing it is _NOT_ in the Python core. Another post in this thread outlined that the first thing you should do is develop something and get people in the community to use it. Please do that, start its own mailing list, and stop discussing it here. On a personal note, I believe that "those who do not understand twisted are doomed to repeat it". I predict that the ultimate outcome of this project will be that all concerned will realize that, actually, Twisted already does almost everything that it proposes, and the microthreading features being discussed here are a trivial hack somewhere in its mainloop machinery, not an entirely new subsystem that it should be implemented in terms of.
glyph@divmod.com wrote:
I have no problem with other, competing event-driven mechanisms being developed;
The need for different event-driven mechanisms to compete with each other is the very problem that needs to be addressed. If Twisted is designed so that it absolutely *has* to use its own special event mechanism, and everything else needs to be modified to suit its requirements, then it's part of the problem, not part of the solution. -- Greg
On Wed, 14 Feb 2007 15:20:13 +1300, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Greg, productive discussion is not furthered by the unsupported statement of one position or another. Instead of only stating what you believe to be a problem, explain why you believe it is a problem. A sentence like:
The need for different event-driven mechanisms to compete with each other is the very problem that needs to be addressed.
Invites a response which merely contradicts it (for example, "you are wrong"), an exchange which hasn't helped anyone to understand anything better. If you present supporting evidence for the position, then the validity and the weight of that evidence can be discussed, and one position or another might be shown to have greater validity. Also, show that you have fully understood the position you are arguing against. For example, if you respond to a message in which someone claims to welcome something, don't respond by saying that requiring that thing is bad. As you know, welcoming something is not the same as requiring that thing, so by making this statement alone, you give the impression of talking past the person to whom you are responding and it may seem to readers that you haven't understood the other person's position.
If Twisted is designed so that it absolutely *has* to use its own special event mechanism, and everything else needs to be modified to suit its requirements, then it's part of the problem, not part of the solution.
Here, you've built on your unsupported premise to arrive at a conclusion which may be controversial. Again, instead of couching the debate in terms of what you might see as self evidence problems, explain why you hold the position you do. That way, the possibility is created for other people to come to understand why you believe the conclusion to be valid. You have presented what could be the beginning of supporting evidence here, in saying that requiring "everything else" to be modified is undesirable. This is only a place to start, not to end, though. You may want to discuss the real scope of modifications required (because "everything" is obviously hyperbole, focusing on what changes are actually necessary would be beneficial) and why you think that modifications are necessary (it may not be clear to others why they are, or it may be the case that others can correct misconceptions you have). For example, you might give a case in which you have needed to integrate Twisted (or a different event framework) with another event loop and describe difficulties you discovered. This will help advance the discussion around practical, specific concerns. Without this focus, it is hard for a discussion to be productive, since it will involve only vague handwaving. Finally, it is often beneficial to avoid bringing up phrases such as "the problem". Particularly in a context such as this, where the existing discussion is focusing on a specific issue, such as the necessity or utility of adding a new set of functionality to the Python standard library, the relevance of "the problem" may not be apparent to readers. In this case, some may not find it obvious how a third party library can be "the problem" with such new functionality. If you explicitly spell out the detrimental consequences of an action, instead of waving around "the problem", the resulting discussion can be that much more productive and focused. Thanks Jean-Paul
Jean-Paul Calderone wrote:
Greg, productive discussion is not furthered by the unsupported statement of one position or another.
Sorry, I'll expand on that a bit. The "problem" I was referring to is the turf wars between all the event-driven frameworks that all want to do things their own way. Solving that is going to require them to give up control and be willing to agree on a standard. It seems reasonable to consider making a suitably enhanced version of asyncore the standard, because it's small and it's already in the stdlib. Someone seemed to be claiming that it would be too hard to rework Twisted's event loop to use anything else, so we had better just use Twisted instead of asyncore, which sounded like a turf-war kind of statement. (That's what I meant by "part of the problem" -- I should really have said it's an *example* of the problem.) From a later post, it seems the main concern is that expanding asyncore to the point where it could support Twisted would involve reinventing most of Twisted's event loop implementation. That's a reasonable point to make, and I don't see anything wrong with examining Twisted's implementation to see what can be used. Whether to adopt Twisted's *API* is a separate issue, though (see next post). -- Greg
On 2/13/07, glyph@divmod.com <glyph@divmod.com> wrote:
Tristan is correct: this should be a patch against Twisted, or perhaps as a separate library that could implement a reactor.
I think there is some confusion here. I am not writing a competing event driven mechanism. What I was doing was feeling out whether there was any interest in better support for asynchronous calls. You are right. This should not have been on this list. It should have been on python-ideas, but I replied here because Dustin started the discussion here and I thought if a generator based microthreading solution were to be implemented, it could also benefit from better asynchronous support so brought it into the discussion. After all, if microthreading was in the Python distribution, then it makes sense to make other general improvements which make them more useful. What I meant to suggest that Twisted could be based on was the asynchronous file IO support using IOCP. Especially because Tristan implied that Twisted's own IOCP reactor was incomplete. I did not say this clearly. Yes I have looked at Twisted. It was the first place I looked, several months ago, and what drew me to it was the IOCP reactor. However as I also explained in my reply to Tristan it is not suitable for my needs. It is a large dependency and it is a secondary framework. And I did not feel the need to verify the implication that it wasn't ready because this matched my own recollections. But I hope you realise that asserting that things should be within Twisted without giving any reason, especially when the first person saying it just stated that the matching work in Twisted wasn't even ready, feels like Twisted is trying to push itself forward by bullying people to work within it whenever something can be asserted as laying within whatever domain it is which Twisted covers. Even if it doesn't suit their needs. I don't think Tristan intended that, but when followed up with your reply and JP's interesting response to Greg, it feels like it.
* Richard Tew <richard.m.tew@gmail.com> [2007-02-14 16:49:03 +0000]:
I am not writing a competing event driven mechanism. What I was doing was feeling out whether there was any interest in better support for asynchronous calls.
I interpreted your suggestions as being about enhancing asyncore with IOCP (or similar); sure, asyncore is already a "competing" event driven mechanism, but for all intents and purposes it is "dead" as there is not much in the way of active development on it, and a relatively small remaining user base. You seemed to be suggesting that asyncore be revived by extending the implementation, as well as sprinkling some generator / tasklets / channels / etc. sugar on top of it.
But I hope you realise that asserting that things should be within Twisted without giving any reason, especially when the first person saying it just stated that the matching work in Twisted wasn't even ready, feels like Twisted is trying to push itself forward by bullying people to work within it whenever something can be asserted as laying within whatever domain it is which Twisted covers.
I don't really think it's "bullying" to suggest that improving Twisted, rather than reimplementing Twisted's core with a brand new implementation slapped straight into the Python stdlib, is counterproductive.
Even if it doesn't suit their needs. I don't think Tristan intended that, but when followed up with your reply and JP's interesting response to Greg, it feels like it.
The gist of my message was "Instead of reimplementing the Twisted reactor, probably poorly, just use Twisted", based on my understanding of what your suggestion was. -- mithrandi, i Ainil en-Balandor, a faer Ambar
Tristan Seligmann <mithrandi-python-dev@mithrandi.za.net> wrote:
* Richard Tew <richard.m.tew@gmail.com> [2007-02-14 16:49:03 +0000]:
I am not writing a competing event driven mechanism. What I was doing was feeling out whether there was any interest in better support for asynchronous calls.
I interpreted your suggestions as being about enhancing asyncore with IOCP (or similar); sure, asyncore is already a "competing" event driven mechanism, but for all intents and purposes it is "dead" as there is not much in the way of active development on it, and a relatively small remaining user base. You seemed to be suggesting that asyncore be revived by extending the implementation, as well as sprinkling some generator / tasklets / channels / etc. sugar on top of it.
It's not dead; much code is still being developed for it, and I recently stepped up as its new maintainer. One of the reasons it's not getting huge new feature additions is generally because it has all of the functionality necessary to get someone started in asynchronous socket programming. That's its purpose. Could it be expanded to support IOCP on Windows? Sure. Is it necessary for its survival? No. - Josiah
participants (6)
-
glyph＠divmod.com -
Greg Ewing -
Jean-Paul Calderone -
Josiah Carlson -
Richard Tew -
Tristan Seligmann