[Python-Dev] More optimisation ideas

[Brett Cannon]

So freezing the stdlib helps on UNIX and not on OS X (if my old testing is
still accurate). I guess the next question is what it does on Windows and
if we would want to ever consider freezing the stdlib as part of the build
process (and if we would want to change the order of importers on
sys.meta_path so frozen modules came after file-based ones).

On Sun, 31 Jan 2016, 10:43 M.-A. Lemburg <mal at egenix.com> wrote:

> On 30.01.2016 20:15, Steve Dower wrote:
> > Brett tried freezing the entire stdlib at one point (as we do for parts
> of importlib) and reported no significant improvement. Since that rules out
> code compilation as well as the OS calls, it'd seem the priority is to
> execute less code on startup.
> >
> > Details of that work were posted to python-dev about twelve months ago,
> IIRC. Maybe a little longer.
>
> Freezing the entire stdlib does improve the startup time,
> simply because it removes stat calls, which dominate the startup
> time at least on Unix.
>
> It also allows sharing the stdlib byte code in memory, since it gets
> stored in static C structs which the OS will happily mmap into
> multiple processes for you without any additional effort.
>
> Our eGenix PyRun does exactly that. Even though the original
> motivation is a different one, the gained improvement in
> startup time is a nice side effect:
>
>     http://www.egenix.com/products/python/PyRun/
>
> Aside: The encodings don't really make much difference here. The
> dictionaries aren't all that big, so generating them on the fly doesn't
> really create much overhead. The trade off in terms of
> maintainability/speed
> definitely leans toward maintainability. For the larger encoding
> tables we already have C implementations with appropriate data
> structures to make lookup speed vs. storage needs efficient.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Jan 31 2016)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> Python Database Interfaces ...           http://products.egenix.com/
> >>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
> ________________________________________________________________________
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>                       http://www.malemburg.com/
>
>
> > Top-posted from my Windows Phone
> >
> > -----Original Message-----
> > From: "Serhiy Storchaka" <storchaka at gmail.com>
> > Sent: ‎1/‎30/‎2016 10:22
> > To: "python-dev at python.org" <python-dev at python.org>
> > Subject: Re: [Python-Dev] More optimisation ideas
> >
> > On 30.01.16 18:31, Steve Dower wrote:
> >> On 30Jan2016 0645, Serhiy Storchaka wrote:
> >>> $ ./python -m timeit -s "import codecs; from encodings.cp437 import
> >>> decoding_table" -- "codecs.charmap_build(decoding_table)"
> >>> 100000 loops, best of 3: 4.36 usec per loop
> >>>
> >>> Getting rid from charmap_build() would save you at most 4.4
> microseconds
> >>> per encoding. 0.0005 seconds if you have imported *all* standard
> >>> encodings!
> >>
> >> Just as happy to be proven wrong. Perhaps I misinterpreted my original
> >> profiling and then, embarrassingly, ran with the result for a long time
> >> without retesting.
> >
> > AFAIK the most time is spent in system calls like stat or open.
> > Archiving the stdlib into the ZIP file and using zipimport can decrease
> > Python startup time (perhaps there is an open issue about this).
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
> >
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> >
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160131/79d2a1be/attachment-0001.html>