[pypy-dev] PGO Optimized Binary

Wang, Peter Xihong peter.xihong.wang at intel.com
Fri Nov 11 15:02:37 EST 2016


Hi Maciej,

This PGO is done for the interpreted code not the JITed code.  Yash would give more details on this.
Part of reason prompted us to do this work is due to 80% of the CPU cycles being spent in interpreted mode, while running OpenStack Swift in a real world scaled cluster setup.  We've got 2% gain from this PGO effort on OpenStack, which is not bad at all.

Thanks,

Peter

 
-----Original Message-----
From: pypy-dev [mailto:pypy-dev-bounces+peter.xihong.wang=intel.com at python.org] On Behalf Of pypy-dev-request at python.org
Sent: Friday, November 11, 2016 9:00 AM
To: pypy-dev at python.org
Subject: pypy-dev Digest, Vol 67, Issue 9

Send pypy-dev mailing list submissions to
	pypy-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://mail.python.org/mailman/listinfo/pypy-dev
or, via email, send a message with subject or body 'help' to
	pypy-dev-request at python.org

You can reach the person managing the list at
	pypy-dev-owner at python.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of pypy-dev digest..."


Today's Topics:

   1. Re: PGO Optimized Binary (Maciej Fijalkowski)
   2. speed.pypy.org and old versions (Matti Picus)
   3. release tarballs of the latest PyPy2.7 available on bitbucket
      (Matti Picus)
   4. Re: speed.pypy.org and old versions (Carl Friedrich Bolz)


----------------------------------------------------------------------

Message: 1
Date: Thu, 10 Nov 2016 20:42:37 +0200
From: Maciej Fijalkowski <fijall at gmail.com>
To: "Singh, Yashwardhan" <yashwardhan.singh at intel.com>
Cc: Armin Rigo <armin.rigo at gmail.com>, "pypy-dev at python.org"
	<pypy-dev at python.org>
Subject: Re: [pypy-dev] PGO Optimized Binary
Message-ID:
	<CAK5idxTd841A3r=Q6ZjzF8e5AGYqrzMa8Oyqwep3uF0j4zA1qQ at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi

8% of that is very good if you can reproduce it across multiple runs (there is a pretty high variance I would think).

You can also try running with --jit off. This gives you an indication of the speed of interpreter, which is a part of warmup

On Wed, Nov 9, 2016 at 12:30 AM, Singh, Yashwardhan <yashwardhan.singh at intel.com> wrote:
> Hi Armin,
>
>
> Thanks for your feedback.
> We ran one of the program suggested by you as an example for evaluation:
> cd rpython/jit/tl
> non-pgo-pypy ../../bin/rpython -O2 --source targettlr pgo-pypy 
> ../../bin/rpython -O2 --source targettlr
>
> We got the following results :
> Non-Pgo pypy -
> [Timer] Timings:
> [Timer] annotate                       ---  7.5 s
> [Timer] rtype_lltype                   ---  5.8 s
> [Timer] backendopt_lltype              ---  3.6 s
> [Timer] stackcheckinsertion_lltype     ---  0.1 s
> [Timer] database_c                     --- 19.6 s
> [Timer] source_c                       ---  2.6 s
> [Timer] =========================================
> [Timer] Total:                         --- 39.2 s
>
> PGO-pypy :
> [Timer] Timings:
> [Timer] annotate                       ---  7.6 s
> [Timer] rtype_lltype                   ---  5.1 s
> [Timer] backendopt_lltype              ---  3.1 s
> [Timer] stackcheckinsertion_lltype     ---  0.0 s
> [Timer] database_c                     --- 18.5 s
> [Timer] source_c                       ---  2.3 s
> [Timer] =========================================
> [Timer] Total:                         --- 36.6 s
>
> The delta in performance  between these two is about 8%.
>
> We are working on getting the data to identify the % of interpreted code vs the jited code for both the binaries. We are also working on creating a pull request to get a better feedback on the change.
>
> Regards
> Yash
>
> ________________________________________
> From: Armin Rigo [armin.rigo at gmail.com]
> Sent: Wednesday, November 02, 2016 2:18 AM
> To: Singh, Yashwardhan
> Cc: pypy-dev at python.org
> Subject: Re: [pypy-dev] PGO Optimized Binary
>
> Hi,
>
> On 31 October 2016 at 22:28, Singh, Yashwardhan 
> <yashwardhan.singh at intel.com> wrote:
>> We applied compiler assisted optimization technique called PGO or Profile Guided Optimization while building PyPy, and found performance got improved by up to 22.4% on the Grand Unified Python Benchmark (GUPB) from ?hg clone https://hg.python.org/benchmarks?.  The below result table shows majority of 51 micros got performance boost with 8 got performance regression.
>
> The kind of performance improvement you are measuring involves only
> short- or very short-running programs.  A few years ago we'd have 
> shrugged it off as irrelevant---"please modify the benchmarks so that 
> they run for at least 10 seconds, more if they are larger"---because 
> the JIT compiler doesn't have a chance to warm up.  But we'd also have 
> shrugged off your whole attempt---"PGO optimization cannot change 
> anything to the speed of JIT-produced machine code".
>
> Nowadays we tend to look more seriously at the cold or warming-up 
> performance too, or at least we know that we should look there.  There 
> are (stalled) plans of setting up a second benchmark suite for PyPy 
> which focuses on this.
>
> You can get an estimate of whether you're looking at cold or hot code:
> compare the timings with CPython.  Also, you can set the environment 
> variable  ``PYPYLOG=jit-summary:-`` and look at the first 2 lines to 
> see how much time was spent warming up the JIT (or attempting to).
>
> Note that we did enable PGO long ago, with modest benefits.  We gave 
> up when our JIT compiler became good enough.  Maybe now is the time to 
> try again (and also, PGO itself might have improved in the meantime).
>
>> We?d like to get some input on how to contribute our optimization recipe to the PyPy dev tree, perhaps by creating an item to the PyPy issue tracker?
>
> The best would be to create a pull request so that we can look at your 
> changes more easily.
>
>> In addition, we would also appreciate any other benchmark or real world use based workload as alternatives to evaluate this.
>
> You can take any Python program that runs either very shortly or not 
> faster than CPython.  For a larger example (with Python 2.7):
>
>     cd rpython/jit/tl
>     python ../../bin/rpython -O2 --source targettlr    # 24 secs
>     pypy ../../bin/rpython -O2 --source targettlr        # 39 secs
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


------------------------------

Message: 2
Date: Fri, 11 Nov 2016 11:56:24 +0200
From: Matti Picus <matti.picus at gmail.com>
To: "pypy-dev at python.org" <pypy-dev at python.org>
Subject: [pypy-dev] speed.pypy.org and old versions
Message-ID: <b2ea7e6e-9f7b-0d0f-0fa8-7bcdfbf13a97 at gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

The landing page for speed.pypy.org, and the pull-downs on the other pages, are clogged with old results from previous versions.
It seems the comparison page http://speed.pypy.org/comparison cannot even pull up the database info, and times out, making it useless.
A quick fix could be to remove tags for much of the historical data, say for things more than 3 years old, keeping a smattering of significant versions?
What you all think?
Matti


------------------------------

Message: 3
Date: Fri, 11 Nov 2016 12:46:19 +0200
From: Matti Picus <matti.picus at gmail.com>
To: "pypy-dev at python.org" <pypy-dev at python.org>
Subject: [pypy-dev] release tarballs of the latest PyPy2.7 available
	on bitbucket
Message-ID: <a36ca1db-1f7a-7ee4-0c66-78b3a4b5f67c at gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

https://bitbucket.org/pypy/pypy/downloads

Please download and try them out, if there are no problems we will try for a Sunday release

Matti



------------------------------

Message: 4
Date: Fri, 11 Nov 2016 14:34:30 +0100
From: Carl Friedrich Bolz <cfbolz at gmx.de>
To: Matti Picus <matti.picus at gmail.com>, "pypy-dev at python.org"
	<pypy-dev at python.org>
Subject: Re: [pypy-dev] speed.pypy.org and old versions
Message-ID: <60f1f4dc-d085-42a6-b7f4-75579d03da55 at email.android.com>
Content-Type: text/plain; charset="utf-8"

Hi Matti, 

Sounds good to me! Maybe we can keep the tag info around somewhere else (a file in extradoc?) in case we ever need it again. 

Thanks for doing this! 
Carl Friedrich

On November 11, 2016 10:56:24 AM GMT+01:00, Matti Picus <matti.picus at gmail.com> wrote:
>The landing page for speed.pypy.org, and the pull-downs on the other 
>pages, are clogged with old results from previous versions.
>It seems the comparison page http://speed.pypy.org/comparison cannot 
>even pull up the database info, and times out, making it useless.
>A quick fix could be to remove tags for much of the historical data, 
>say for things more than 3 years old, keeping a smattering of 
>significant versions?
>What you all think?
>Matti
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20161111/a29f5816/attachment-0001.html>

------------------------------

Subject: Digest Footer

_______________________________________________
pypy-dev mailing list
pypy-dev at python.org
https://mail.python.org/mailman/listinfo/pypy-dev


------------------------------

End of pypy-dev Digest, Vol 67, Issue 9
***************************************


More information about the pypy-dev mailing list