[Python-ideas] fork

[Sven R. Kunze]

On 06.08.2015 17:52, Xavier Combelle wrote:
>
>>     One quick comment: from my experience (mostly with other
>>     languages that are very different from Python, so I can't promise
>>     how well it applies here...), implicit futures without implicit
>>     laziness or even an explicit delay mechanism are not as useful as
>>     they look at first glance. Code that forks off 8 Fibonacci calls,
>>     but waits for each one's result before forking off the next one,
>>     might as well have just stayed sequential. And if you're going to
>>     use the result by forking off another job, then it's actually
>>     more convenient to use explicit futures like the ones in the stdlib.
>>
>>     One slightly bigger idea: If you really want to pursue your
>>     implicit-as-possible design further, you might want to consider
>>     making the decorators replace the function with an object whose
>>     __call__ method just implicitly submits it to the pool.
>
>     I added two new decorators for this. But they don't work with the
>     @ syntax. It seems like a well-known issue of Python:
>
>     _pickle.PicklingError: Can't pickle <function fib_fork at
>     0x7f8eaeb09730>: it's not the same object as __main__.fib_fork
>
>     Would be great if somebody could fix that.
>
>
> Sorry but I don't follow you have you any example that fail ?

I fixed that, well, halfhearted:

https://github.com/srkunze/fork/blob/2359265/fork.py#L47 and the 
following 3 lines.

Remove that lines, and the tests using @cpu_bound_fork will fail.

The reason for this is that the Pickle module is only capable of 
pickling module-level-named objects.


Do you have a better fix? I would rather see that fixed in the Python 
internal decorator implementation than by me.


Cheers,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150806/13828a75/attachment-0001.html>