Errhm...., On 20.10.12 00:31, Christian Tismer wrote:
Hi Guido, Marc, all,
this is a veery promising result, telling me that the big Oh can in fact be neglected in real applications. 20 for twi is good!
I will of course do an analysis and find the parameters of the quadratic, but my concern is pretty much tamed.
For me that means there will soon be a library that contains real generators and more building blocks.
I think using those would simplify the design of the async API quite a lot.
I suggest to regard current generator constructs as low-level helpers for Implementing the real concurrency building blocks.
Instead of using the existing re.compile("yield (from)?") pattern, I think we can abstract from this now and think in terms of higher level constructs.
Let's assume generators and coroutines, and model concurrency from that. I believe this unwinds the brains and clarifies things a lot.
I will provide sone classes for that at the pycon.de sprint, unless somebody implements it earlier (please don't).
This email was written in non-linear order, so please ignore logic inversions.
Cheers - chris
Sent from my Ei4Steve
On Oct 19, 2012, at 23:31, Guido van Rossum
wrote: On Fri, Oct 19, 2012 at 2:15 PM, Mark Shannon wrote:
Hi Nick,
On 16.10.12 03:49, Nick Coghlan wrote:
On Tue, Oct 16, 2012 at 10:44 AM, Greg Ewing
wrote: My original implementation of yield-from actually *did* avoid this, by keeping a C-level pointer chain of yielding-from frames. But that part was ripped out at the last minute when someone discovered that it had a detrimental effect on tracebacks.
There are probably other ways the traceback problem could be fixed, so maybe we will get this optimisation back one day. Ah, I thought I remembered something along those lines. IIRC, it was a bug report on one of the alphas that prompted us to change it. I was curious and searched quite a lot. It was v3.3.0a1 from 2012-03-15 as a reaction to #14230 and #14220 from Marc Shannon, patched by Benjamin.
Now I found the original implementation. That looks very much as I'm thinking it should be.
Quite a dramatic change which works well, but really seems to remove what I would call "now I can emulate most of Stackless" efficiently.
Maybe I should just try to think it would be implemented as before, build an abstraction and just use it for now.
I will spend my time at PyCon de for sprinting on "yield from". The current implementation may not be much slower than Greg's original version. One of the main costs of making a call is the creation of a new
On 19/10/12 13:55, Christian Tismer wrote: frame. But calling into a generator does not need a new frame, so the cost will be reduced. Unless anyone has evidence to the contrary :)
Rather than increasing the performance of this special case, I would suggest that improving the performance of calls & returns in general would be a more worthwhile goal. Calls and returns ought to be cheap. I did a basic timing test using a simple recursive function and a recursive PEP-380 coroutine computing the same value (see attachment). The coroutine version is a little over twice as slow as the function version. I find that acceptable. This went 20 deep, making 2 recursive calls at each level (except at the deepest level).
Output on my MacBook Pro:
plain 2097151 0.5880069732666016 coro. 2097151 1.2958409786224365
This was a Python 3.3 built a few days ago from the 3.3 branch.
What you are comparing seems to have a constant factor of about 2.5. minimax:py3 tismer$ python3 p3time.py plain 0 1 0.00000 coro. 0 1 0.00001 relat 0 1 8.50000 plain 1 3 0.00000 coro. 1 3 0.00001 relat 1 3 2.77778 plain 2 7 0.00000 coro. 2 7 0.00001 relat 2 7 3.62500 plain 3 15 0.00000 coro. 3 15 0.00001 relat 3 15 2.87500 plain 4 31 0.00001 coro. 4 31 0.00002 relat 4 31 2.42424 plain 5 63 0.00002 coro. 5 63 0.00004 relat 5 63 2.46032 plain 6 127 0.00003 coro. 6 127 0.00007 relat 6 127 2.52542 plain 7 255 0.00006 coro. 7 255 0.00014 relat 7 255 2.38272 plain 8 511 0.00011 coro. 8 511 0.00028 relat 8 511 2.49356 plain 9 1023 0.00022 coro. 9 1023 0.00055 relat 9 1023 2.50327 plain 10 2047 0.00042 coro. 10 2047 0.00106 relat 10 2047 2.50956 plain 11 4095 0.00083 coro. 11 4095 0.00204 relat 11 4095 2.44699 plain 12 8191 0.00167 coro. 12 8191 0.00441 relat 12 8191 2.64792 plain 13 16383 0.00340 coro. 13 16383 0.00855 relat 13 16383 2.51881 plain 14 32767 0.00876 coro. 14 32767 0.01823 relat 14 32767 2.08106 plain 15 65535 0.01419 coro. 15 65535 0.03507 relat 15 65535 2.47131 plain 16 131071 0.02669 coro. 16 131071 0.06874 relat 16 131071 2.57515 plain 17 262143 0.05448 coro. 17 262143 0.13699 relat 17 262143 2.51467 plain 18 524287 0.10843 coro. 18 524287 0.27395 relat 18 524287 2.52660 plain 19 1048575 0.21310 coro. 19 1048575 0.54573 relat 19 1048575 2.56095 plain 20 2097151 0.42802 coro. 20 2097151 1.06199 relat 20 2097151 2.48114 plain 21 4194303 0.86531 coro. 21 4194303 2.19048 relat 21 4194303 2.53143 ciao - chris -- Christian Tismer :^) mailto:tismer@stackless.com Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/