[Python-Dev] Pure pickle bechmark.

[INADA Naoki]

I said about pure Python implementation (unpickle_pure_python),
because mail title is "Pure pickle bechmark".
INADA Naoki  <songofacandy at gmail.com>


On Mon, Jul 10, 2017 at 8:36 AM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Wait. Are we talking about the C accelerator or the pure Python
> implementation of pickle on Python 3?
>
> Victor
>
> Le 10 juil. 2017 01:19, "INADA Naoki" <songofacandy at gmail.com> a écrit :
>>
>> I don't know this is relating to your case.
>>
>> When I saw Victor's report [1], I researched why Python 3 is slower than
>> Python 2 on unpickle_pure_python benchmark.
>>
>> [1] https://mail.python.org/pipermail/speed/2017-February/000503.html
>>
>>
>> And I found Python 2 and 3 uses different version of pickle format.
>>
>> Current Python 3 uses "framing" format.  While unpickling, `read(1)` is
>> very performance critical.  Python 2 uses `cStringIO.read` which is
>> implemented in C.
>> On the other hand, Python 3 uses `_Unframer.read` which is implemented
>> in Python.
>>
>> Since this is not relating to "first import time", I don't know this
>> is what you want to optimize.
>> (Since _pickle is used for normal case, pure Python unpickle
>> performance is not a common
>> problem).
>>
>> If you want to optimize it, _Unframer uses BytesIO internally and
>> performance critical
>> part may be able to call BytesIO.read directly instead of _Unframer.read.
>>
>> Regards,
>> INADA Naoki  <songofacandy at gmail.com>
>>
>>
>> On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya <bhavishyagopesh at gmail.com>
>> wrote:
>> > Hello,
>> >
>> > 1).I was going through the code of python pickle to search any
>> > optimization
>> > possibility.But the only thing that I found very alarming was again the
>> > import time(I tried with lazy-import but it didn't helped much.)
>> >
>> > I found py3 to be ~45 times slower on initial imports(very raw
>> > measure..using "time." ) as compared to py2 on an usual example.
>> >
>> > py3->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.009715557098388672(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > py2->
>> > ./python -c '
>> > favorite_color = { "lion": "yellow", "kitty": "red" }
>> > pickle.dump( favorite_color, open( "save.p", "wb" ) )'
>> > 0.000236034393311(time taken to do initial imports...measured using
>> > time.time() )
>> >
>> > Do you have any thought/ideas on improving this?
>> >
>> >
>> > Thank You.
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com