> What are you referring to as a "coroutine" here?  A generator?

Generator, yes, but used as a coroutine that both emits values and receives them. The introduction of dedicated coroutines in Python 3.5 is something I haven't updated my personal language for very well. Generator-coroutine is the term used in there Python docs, IIRC.

> And exponential growth, you say?

Well... That may not be perfectly accurate and I shouldn't have used that term. It grew large enough, fast enough that the same level of nesting was measured in microseconds for the control and and over an hour for the wrapper so I cancelled the test. Let me put together a reproducible experiment and let others decide the growth rate.

> Where is this implemented?

Unfortunately, our production use of the lib is not something we've open sourced. Similar to the above, I'll make available a reproducible experiment so you can draw your own conclusion.

> Optimizing for CPython looks like adding weird implementation-specific hacks that might start working or be backwards in the next version

This went into the rationale behind making most of the components pluggable. The choice of optimizing for cPython is internal to us and we chose to document it for others. It's not required and not the default.


As far as the possibility of integrating this feature into Twisted, let me chat with my cohorts over here. Well come back with some benchmark code and a proposal of changes for further discussion. I'll probably start a new thread to focus on the context bits.

On Wed, Oct 19, 2016, 16:56 Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring <itamar@itamarst.org> wrote:

On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
Well... I had a test that went through synchronous Deferred path. And yeah, it was easier to write than async test. But it failed to catch a bug that was only in async case. So the problem I see is that supporting both in Deferred means you need twice the number of tests each time you use Deferreds.

Er, that was unclear. I had a bug that wasn't caught by tests because it passed with sync Deferred and failed with async Deferred callback, and I didn't have tests for latter.

To be clear: I do see this as a downside to Deferred's architecture; it's a tradeoff.  I see the loose coupling with the event loop as a worthwhile upside.

However, it's totally possible to write the async Deferred callback case as well, just by having the test fire the Deferred after returning to the test body instead of firing it before :-).

-glyph
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python