<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 04.08.2015 05:21, Andrew Barnert wrote:<br>
<blockquote
cite="mid:6A8EA952-ED98-4C26-9A40-54BE54367849@yahoo.com"
type="cite">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div>On Aug 3, 2015, at 10:11, Sven R. Kunze <<a
moz-do-not-send="true" href="mailto:srkunze@mail.de">srkunze@mail.de</a>>
wrote:</div>
<blockquote type="cite">
<div> <br>
<div class="moz-cite-prefix">On 02.08.2015 02:02, Andrew
Barnert wrote:<br>
</div>
<blockquote
cite="mid:D897A956-964A-4036-9423-922A5635C72B@yahoo.com"
type="cite">
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
<div>Your idea of having a single global "pool manager"
object, where you could submit tasks and, depending on how
they're marked, they get handled differently might have
merit. But that's something you could build pretty easily
on top of concurrent.futures (at least for threads vs.
processes; you can add in coroutines later, because
they're not quite as easy to integrate), upload to PyPI,</div>
</blockquote>
<br>
You mean something like this?<br>
<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://pypi.python.org/pypi/xfork">https://pypi.python.org/pypi/xfork</a><br>
</div>
</blockquote>
<div><br>
</div>
Did you just write this today? Then yes, that proves my point
about how easy it is to write it. Now you just have to get people
using it, get some experience with it, etc. and you can come back
with a proposal to put something like this in the stdlib, add
syntactic support, etc. that it will be hard for anyone to
disagree with. (Or to discover that it has flaws that need to be
fixed, or fundamental flaws that can't be fixed, before making the
proposal.)</blockquote>
<br>
I presented it today. The team members already showed interest. They
also noted they like its simplicity. The missing syntax support
seemed like minor issue compared to what complexity is hidden.<br>
<br>
Others admitted they knew about the existence of concurrent.futures
and such but never used it due to<br>
- its complexity<br>
- AND *drum roll* the '.result()' of the future objects<br>
As it seems, it doesn't feel natural.<br>
<br>
<blockquote
cite="mid:6A8EA952-ED98-4C26-9A40-54BE54367849@yahoo.com"
type="cite">
<div>One quick comment: from my experience (mostly with other
languages that are very different from Python, so I can't
promise how well it applies here...), implicit futures without
implicit laziness or even an explicit delay mechanism are not as
useful as they look at first glance. Code that forks off 8
Fibonacci calls, but waits for each one's result before forking
off the next one, might as well have just stayed sequential. And
if you're going to use the result by forking off another job,
then it's actually more convenient to use explicit futures like
the ones in the stdlib.</div>
<div><br>
</div>
<div>One slightly bigger idea: If you really want to pursue your
implicit-as-possible design further, you might want to consider
making the decorators replace the function with an object whose
__call__ method just implicitly submits it to the pool.</div>
</blockquote>
<br>
I added two new decorators for this. But they don't work with the @
syntax. It seems like a well-known issue of Python:<br>
<br>
_pickle.PicklingError: Can't pickle <function fib_fork at
0x7f8eaeb09730>: it's not the same object as __main__.fib_fork<br>
<br>
Would be great if somebody could fix that.<br>
<br>
<blockquote
cite="mid:6A8EA952-ED98-4C26-9A40-54BE54367849@yahoo.com"
type="cite">
<div>Then you can use normal function-calling syntax and pretend
everything is magic. You can even add operator dunder methods to
your future class that do the same thing (so "result * 2" just
builds a new future out of "self.get() * 2", either submitted to
the pool, probably better, tacked on as an add_done_callback). I
think there's a limit to how far you can push this without some
mechanism to mark when you need to actual value (in ML-derived
languages and C++, static types make this easier: a cast,
implicit or explicit, forces a wait; in Python, that doesn't
work), but it might be worth exploring that limit. Or it might
be better to just stop at the magic function calls and leave the
futures alone.</div>
</blockquote>
<br>
I actually like the idea of contagious futures and I might outline
why this is not an issue with the current Python language.<br>
<br>
Have a look at the following small interactive Python session:<br>
<br>
>>> 3+4<br>
7<br>
>>> _<br>
7<br>
>>> a=9<br>
>>> _<br>
7<br>
>>> a+=10<br>
>>> _<br>
7<br>
>>> a<br>
19<br>
>>> _<br>
19<br>
>>> <br>
<br>
<br>
Question:<br>
When has the add operation being executed?<br>
<br>
Answer:<br>
Unknown from the programmer's perspective.<br>
<br>
Only requirement:<br>
Exceptions are raised exactly where the operation is supposed to
take place in the source code (even if the operation that raises the
exception is performed later).<br>
<br>
<br>
Best,<br>
Sven<br>
</body>
</html>