[Python-Dev] Twisted Isn't Specific (was Re: Trial balloon: microthreads library in stdlib)

[Jean-Paul Calderone]

On Thu, 15 Feb 2007 15:47:39 +1300, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>Steve Holden wrote:
>
>> If the borrowed code takes a reactor parameter then presumably the
>> top-level code can pass the appropriate reactor type in.
>
>Since there should only be one reactor at a time in
>any given application, it shouldn't have to be passed
>in -- it could be held in a global variable deep
>inside the library. Only the code which creates the
>reactor initially needs to know about that variable,
>or even that there is such a thing as a reactor.

Whether or not the premise here is accurate may be out
of scope for this thread.  Or it may not be.  I dunno.
However, I do want to point out that it is not necessarily
correct that there should be only one reactor at a time in
a given application.  PJE has already explained that
peak.events can have multiple reactors.  Twisted is tied to
one, but this may not always be the case.  Whether there is
a default reactor for applications that don't care about the
ability to have more than one at a time is yet another
question which may be worth examining.  These are the kinds
of things which should be spelled out in a PEP, including
the rationale for any particular policy decisions (which
should be kept to an absolute minimum) are made.

>
>> "Incorporating some piece of event-driven code written by someone else"
>> implies specific assumptions about event types and delivery, surely.
>
>It requires agreement on how to specify the event types
>and what to do in response, but that's all it should
>require.
>
>The way I envisage it, setting up an event callback
>should be like opening a file -- there's only one way
>to do it, and you don't have to worry about what the
>rest of the application is doing. You don't have to
>get passed an object that knows how to open files --
>it's a fundamental service provided by the system.
>You just use it.

If we suppose that files and sockets are supported in
roughly the same way, and we suppose that sockets are
supported in the way that Twisted supports them, then
there is no difficulty supporting files in this way. :)

>
>> That's why it's difficult to port code between GUI toolkits, for
>> example, and even more so to write code that runs on several toolkits
>> without change.
>
>Just in case it's not clear, the events I'm talking
>about are things like file and socket I/O, not GUI
>events. Trying to use two different GUIs at once is
>not something I'm addressing.

Alright, good.  Getting two different GUI libraries to
play together is a pretty hairy task indeed, and well
worth keeping separate from this one. :)

>
>Rather, you should be able to write code that does
>e.g. some async socket I/O, and embed it in a GUI
>app using e.g. gtk, without having to modify it to
>take account of the fact that it's working in a
>gtk environment, or having to parameterise it to
>allow for such things.

Excellent.  To be clear, this is how the Twisted model
works, with respect to integration with GUI toolkits.
I would not enjoy working with a system in which this
was not the case.

>
>> You seem to be arguing for libraries that contain platform dependencies
>> to handle multiple platforms.
>
>I'm arguing that as much of the platform dependency
>as possible should be in the asyncore library (or
>whatever replaces it).

Certainly.  Library code doesn't care if the event
loop is driven by select or poll or epoll or /dev/poll
or kqueue or aio or iocp or win32 events or realtime
signals or kaio or whatever gnarly thing is hidden in
gtk or whatever gnarly thing is hidden inside qt or
whatever gnarly thing is hidden inside COM or whatever
gnarly thing is hidden inside wxWidgets.  It cares
about what features are available.  It requests them
somehow, and uses them.  If they are unavailable, then
it can decide whether the lack is catastrophic and give
up or if it can be worked around somehow.  The way a
Twisted application does this is based on interfaces.
Assuming interfaces continue to not be present in the
stdlib, a stdlib event loop would have to find some
other API for presenting this information, but it is not
a very hard problem to solve.

>The main application code
>*might* have to give it a hint such as "this app
>uses gtk", but no more than that. And ideally, I'd
>prefer it not to even have to do that -- pygtk
>should do whatever is necessary to hook itself
>into asyncore if at all possible, not the other
>way around.

There is some advantage to declaring things up front,
lest you get into the situation where you are partway
through using code which will suddenly begin to demand
Gtk at the same time as you are partway through using
code which will suddenly begin to demand Qt, at which
point you are in trouble.  But this is another minor
point.

>
>> Since Glyph has already stated his opinion that Twisted isn't yet ready
>> for adoption as-is this doesn't add to the discussion.
>
>Okay, but one of the suggestions made seemed to be
>"why not just use the Twisted API". I'm putting
>forward a possible reason.

So far, it sounds like the objections to using the
Twisted API are primarily based on misunderstandings
of it, and the actual desired API looks pretty much
the same as Twisted's.

Jean-Paul