[Python-ideas] Do not block threads when pickle/unpickle

[Martin Bammer]

If you've for example an application with a GUI. Every time bigger 
objects are pickled/unpickled the complete GUI blocks.

Or if you've a server application with multiple network connections and 
the application should have a guaranteed response time, then it is 
impossible if one single client could lead the server to block for a 
longer time if it sends a big object which takes long to unpickle.

Just 2 simple examples.


Am 2018-07-16 um 20:05 schrieb Joseph Jevnik:
> The GIL must be held to allocate memory for Python objects and to
> invoke the Python code to deserialize user defined picklable objects.
> I don't think there is a long span of time where the code could leave
> the GIL released. The Python implementation is just pausing to let
> other Python threads run, but it is not actually able to parallelize.
> The same would be true of the C implementation, is there a reason to
> want to pause the thread to let another thread run?
>
> On Mon, Jul 16, 2018 at 1:56 PM, Martin Bammer <mrbm74 at gmail.com> wrote:
>> Hi,
>>
>> the old and slow python implementation of pickle didn't block background
>> thread.
>>
>> But the newer C-implementation blocks other threads while dump/load is
>> running.
>>
>> Wouldn't it be possible to allow other threads during this time?
>>
>> Especially could load/loads release the GIL, because Python objects are not
>> available to the Python code until these functions have finished?
>>
>> Regards,
>>
>> Martin
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/