[Python-Dev] first post introduction and question regarding lto

[Soldea, Octavian]

Hello

Thank you for your messages. I am not quite sure I understood, therefore, I would like to ask if you see beneficial to have the option of lto separately?

Best regards,
  Octavian






-----Original Message-----
From: Wang, Peter Xihong 
Sent: Monday, August 7, 2017 5:00 PM
To: Victor Stinner <victor.stinner at gmail.com>; Soldea, Octavian <octavian.soldea at intel.com>
Cc: Python-Dev at python.org
Subject: RE: [Python-Dev] first post introduction and question regarding lto

We evaluated different tests before setting down and proposing regrtest suite for PGO training, including using OpenStack benchmarks for OpenStack applications.  The regrtest suite was found consistently giving the best in terms of performance across applications/workloads. 

So I would hesitate to remove any test from the training suite (for the purpose of reducing build time), unless we are absolute sure it will not hurt performance across the board.

Thanks,

Peter

-----Original Message-----
From: Python-Dev [mailto:python-dev-bounces+peter.xihong.wang=intel.com at python.org] On Behalf Of Victor Stinner
Sent: Monday, August 07, 2017 4:43 PM
To: Soldea, Octavian <octavian.soldea at intel.com>
Cc: Python-Dev at python.org
Subject: Re: [Python-Dev] first post introduction and question regarding lto

I don't think that PGO compilation itself is slow. Basically, I expect that it only doubles the compilation time, but compiling Python takes less than 1 minute usually. The slow part is the profiling task: run the full Python test suite, which takes at least 20 minutes. The tests must be run sequentially.

It would help to reduce the number of tests run in the profiling task.
We should also maybe skip tests which depend too much on I/O to get more reproductible PGO performances.

Victor

2017-08-08 1:01 GMT+02:00 Soldea, Octavian <octavian.soldea at intel.com>:
> Hello
>
>
>
> Gregory: Thank you for your message. I really appreciate it.
>
>
>
> In my opinion the use of lto only compilation mode can be of benefit 
> from two reasons at least:
>
> 1.       Lto only can provide a platform for experimentations. Of course, it
> seems to be very application specific.
>
> 2.       Lto compilation is faster than an entire pgo + lto mode. While pgo
> can take a lot of time, lto can provide significant optimizations, as 
> compared to baseline, at the cost of less compilation time. Of course, 
> lto is time consuming. To the best of my knowledge, compiling with lto 
> only is shorter than combining pgo and lto.
>
>
>
> In this context, I will be more than happy to receive more feedback 
> and opinions.
>
>
>
> Best regards,
>
>   Octavian
>
>
>
>
>
>
>
>
>
> From: Gregory P. Smith [mailto:greg at krypto.org]
> Sent: Monday, August 7, 2017 3:12 PM
> To: Soldea, Octavian <octavian.soldea at intel.com>; 
> Python-Dev at python.org
> Subject: Re: [Python-Dev] first post introduction and question 
> regarding lto
>
>
>
> I've personally never seen a situation where PGO is not desired yet 
> some other fancier optimization such as LTO is. When do you encounter 
> people wanting that?  PGO always produces a 10-20% faster CPython interpreter.
>
>
>
> I have no problem with patches enabling an LTO only build for anyone 
> who wants one if they do not already exist.  At the moment I'm not 
> sure which LTO toolchains actually work properly.  We removed it from 
> inclusion in --enable-optimizations because it was clearly broken in 
> some common environments.
>
>
>
> If LTO without PGO is useful, making it work is probably just a matter 
> of making sure @LTOFLAGS@ show up on the non-PGO related 
> Makefile.pre.in targets as well as updating the help text for --with-lto in configure.ac.
> [untested]
>
>
>
> -gps
>
>
>
>
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gm
> ail.com
>
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/peter.xihong.wang%40intel.com