[Python-Dev] threadsafe patch for asynchat

[Tim Peters]

[Josiah Carlson]
> ...
> Back to the topic that Guido was really complaining about: Zope +
> asyncore.  I don't doubt that getting Zope to play nicely with asyncore
> was difficult,

It's more that mixing asyncore with threads is a bloody nightmare, and
ZEO and Zope both do that.  Zope (but not ZEO) goes on to mix threads
with asynchat too.  In addition, ZEO makes life much harder than
should be necessary by running in two different modes and
auto-switching between them, depending on whether "the app" is or is
not running an asyncore mainloop itself.  In order to _detect_ when
"the app" fires up an asyncore mainloop, ZEO monkey-patches asyncore's
loop() function and physically replaces it with its own loop()
function.  It goes downhill from there.

Guido's memories are partly out of date now:  ZEO used to replace a
lot more of asyncore than it does now, because of bugs in the asyncore
distributed with older Python versions.  The _needs_ for that went
away little by little over the years, but the code in ZEO stuck around
much longer.  ZEO's ThreadedAsync/LoopCallback.py is much smaller now
(ZODB 3.6) than Guido remembers.

For a brief while, I even ripped out ZEO's monkey-patching of Python's
asyncore loop(), but it turned out that newer code in Zope3 (but not
Zope2) relied on, in turn, poking values into ZEO's module globals to
cause ZEO's loop() replacement to shut down (that's the kind of
"expedient" joy you get when mixing asyncore with threads).

Every piece of it remains "underdocumented" and, IMO, highly obscure.

> but it begs the questions: what would have been done if asyncore didn't exist,

Who knows?  What would python-dev be like if you didn't exist :-)?

> and why wasn't that done instead of trying to play nicely with asyncore?

Bugs and "missing features" in asyncore.  For ZEO's purposes, if I had
designed it, I expect it would have used threads (without asyncore). 
However, bits of code still sitting around suggest that it was at
least the _intent_ at one time that ZEO be able to run without threads
at all.  That's certainly not possible now.

If you look at asyncore's revision history, you'll note that Jeremy
and Guido made many changes when they worked at Zope Corp.  Those
largely reflect the history of moving ZEO's asyncore monkey-patches
into the Python core.

BTW, if you don't use ZEO, I believe it's possible to run Zope3
without asyncore (you can use Twisted in Zope3 instead).