[Python-Dev] More optimisation ideas

M.-A. Lemburg mal at egenix.com
Sun Jan 31 13:43:20 EST 2016


On 30.01.2016 20:15, Steve Dower wrote:
> Brett tried freezing the entire stdlib at one point (as we do for parts of importlib) and reported no significant improvement. Since that rules out code compilation as well as the OS calls, it'd seem the priority is to execute less code on startup.
> 
> Details of that work were posted to python-dev about twelve months ago, IIRC. Maybe a little longer.

Freezing the entire stdlib does improve the startup time,
simply because it removes stat calls, which dominate the startup
time at least on Unix.

It also allows sharing the stdlib byte code in memory, since it gets
stored in static C structs which the OS will happily mmap into
multiple processes for you without any additional effort.

Our eGenix PyRun does exactly that. Even though the original
motivation is a different one, the gained improvement in
startup time is a nice side effect:

    http://www.egenix.com/products/python/PyRun/

Aside: The encodings don't really make much difference here. The
dictionaries aren't all that big, so generating them on the fly doesn't
really create much overhead. The trade off in terms of maintainability/speed
definitely leans toward maintainability. For the larger encoding
tables we already have C implementations with appropriate data
structures to make lookup speed vs. storage needs efficient.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 31 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


> Top-posted from my Windows Phone
> 
> -----Original Message-----
> From: "Serhiy Storchaka" <storchaka at gmail.com>
> Sent: ‎1/‎30/‎2016 10:22
> To: "python-dev at python.org" <python-dev at python.org>
> Subject: Re: [Python-Dev] More optimisation ideas
> 
> On 30.01.16 18:31, Steve Dower wrote:
>> On 30Jan2016 0645, Serhiy Storchaka wrote:
>>> $ ./python -m timeit -s "import codecs; from encodings.cp437 import
>>> decoding_table" -- "codecs.charmap_build(decoding_table)"
>>> 100000 loops, best of 3: 4.36 usec per loop
>>>
>>> Getting rid from charmap_build() would save you at most 4.4 microseconds
>>> per encoding. 0.0005 seconds if you have imported *all* standard
>>> encodings!
>>
>> Just as happy to be proven wrong. Perhaps I misinterpreted my original
>> profiling and then, embarrassingly, ran with the result for a long time
>> without retesting.
> 
> AFAIK the most time is spent in system calls like stat or open. 
> Archiving the stdlib into the ZIP file and using zipimport can decrease 
> Python startup time (perhaps there is an open issue about this).
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 



More information about the Python-Dev mailing list