Mailman 3 November 2016 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

RPython Darwin platform description so_prefixes empty
by John Zhang 28 Dec '16

28 Dec '16

Hi all, I have been doing some coding using RFFI to load function pointers from shared libraries (`llexternal` function). However what bothers me is that apparently `Darwin_x86_64` platform class (defined in translator/platform/darwin.py) does not include ‘lib’ in `so_prefixes`. This is causing me some problems with not being able to find my shared libraries on macOS X. I’m wondering if that’s intentional, or just a mistake? It should be a simple fix if it’s a mistake. Regards, John Zhang ------------------------------------------------------ John Zhang Research Assistant Programming Languages, Design & Implementation Division Computer Systems Group ANU College of Engineering & Computer Science 108 North Rd The Australian National University Acton ACT 2601 john.zhang(a)anu.edu.au<mailto:john.zhang@anu.edu.au>

2 5

Dontbug: A reversible debugger for PHP (similar in concept to RevDB for Python/PyPy)
by Sidharth Kshatriya 17 Dec '16

17 Dec '16

Dear All, There have been some interesting blogs about RevDB a reversible debugger for Python on the PyPy blog. I'd like to tell you about Dontbug, a reversible debugger for PHP that I recently released. Like RevDB, it allows you to debug forwards and backwards -- but in PHP. See: https://github.com/sidkshatriya/dontbug For a short (1m35s) demo video: https://www.youtube.com/watch?v=DA76z77KtY0 Why am I talking about this in a PyPy mailing list :-) ? Firstly, because I think reverse debuggers for dynamic languages are relatively rare -- so its a good idea that we know about each other! Secondly, the fact that there are more and more reversible debuggers for various languages every year means that reverse debugging is definitely entering the mainstream. We could be at an inflexion point here! Hope you guys find Dontbug interesting! Thanks, Sidharth

3 4

Bump DARWIN_VERSION_MIN to 10.7?
by Jeremy Thurgood 30 Nov '16

30 Nov '16

Hi, I recently tried to build RevDB on OS X and the build failed because __thread isn't supported on OS X 10.6 which RPython specifies as the minimum supported version. I've updated DARWIN_VERSION_MIN to 10.7 in the reverse-debugger branch[1], because otherwise it doesn't build at all on OS X. Armin suggested that there would be a performance benefit to increasing DARWIN_VERSION_MIN for pypy as well (along with adding darwin to the SUPPORT__THREAD list). I would like to make this change, but I don't want to break pypy for anyone who may still need OS X 10.6 support. As far as I can tell, OS X 10.6 is unsupported (last release in 2011) but still available as a necessary upgrade step from older versions that don't support the app store to newer versions that are only available through the app store. I also have an old laptop running OS X 10.8, and several applications (including Chrome) have dropped support for that recently. [1] https://bitbucket.org/pypy/pypy/commits/a740348ea339c358601d9029bc6811f56c4… Thanks, --jerith

2 1

assert error assert sys.maxint == (2**31 - 1)
by Shubha Ramani 30 Nov '16

30 Nov '16

Why ? When I run rpython:/opt/pypy/rpython$ ./bin/rpython -Ojit --jit-backend=x86 ./translator/goal/targettestdicts.py [translation:info] Error: File "/opt/pypy/rpython/translator/goal/translate.py", line 318, in main drv.proceed(goals) File "/opt/pypy/rpython/translator/driver.py", line 551, in proceed result = self._execute(goals, task_skip = self._maybe_skip()) File "/opt/pypy/rpython/translator/tool/taskengine.py", line 114, in _execute res = self._do(goal, taskcallable, *args, **kwds) File "/opt/pypy/rpython/translator/driver.py", line 278, in _do res = func() File "/opt/pypy/rpython/translator/driver.py", line 361, in task_pyjitpl_lltype backend_name=self.config.translation.jit_backend, inline=True) File "/opt/pypy/rpython/jit/metainterp/warmspot.py", line 49, in apply_jit **kwds) File "/opt/pypy/rpython/jit/metainterp/warmspot.py", line 224, in __init__ self.build_cpu(CPUClass, **kwds) File "/opt/pypy/rpython/jit/metainterp/warmspot.py", line 473, in build_cpu translate_support_code, gcdescr=self.gcdescr) File "/opt/pypy/rpython/jit/backend/x86/runner.py", line 143, in __init__ assert sys.maxint == (2**31 - 1)[translation:ERROR] AssertionError[translation] start debugger...> /opt/pypy/rpython/jit/backend/x86/runner.py(143)__init__()-> assert sys.maxint == (2**31 - 1)(Pdb+) Shubha D. Ramanishubharamani(a)gmail.com shubharamani(a)yahoo.com

2 3

Sample benchmarks to run through pypy JIT ?
by Shubha Ramani 29 Nov '16

29 Nov '16

Hello. I'm taking over development of the vtune branch. I do work for Intelbut it's easier to use my personal account. Question, as I develop this branch can someone recommend some good Python benchmarks out there which have been tested with PyPy and have seen performance improvements ? pypy / pypy / vtune — Bitbucket | | | | | | | | | | | pypy / pypy / vtune — Bitbucket | | | | Shubha D. Ramanishubharamani(a)gmail.com shubharamani(a)yahoo.com

2 2

Can't run any puppy tests, get --assert: invalid choice etc...
by Shubha Ramani 27 Nov '16

27 Nov '16

For instance this command :usage: py.test [options] [file_or_dir] [file_or_dir] [...]py.test: error: argument --assert: invalid choice: 'reinterp' (choose from 'rewrite', 'plain') Shubha D. Ramanishubharamani(a)gmail.com shubharamani(a)yahoo.com

2 4

array sorting
by Thomas Conway 27 Nov '16

27 Nov '16

Hi Pypies, I'm new to the list, so let me start by saying I totally love pypy and am using it for my bioinformatics codes. I realise this isn't exactly a pypy specific problem, by the standard array module doesn't provide sort() on arrays. In consequence you either have to convert them to lists and use list.sort() or you have to implement sort on them yourself. I've implemented several sorts (radix, american flag, quick sort, in-place heap sort) and they all perform slower than converting to a list and sorting that. It strikes me that any python implementation should benefit from an array.sort() method since unboxing/type checking is unnecessary. What is the wisdom of this community on the matter? It's something I'd consider taking on if it was generally deemed worthwhile. Tom. -- Thomas Conway drtomc(a)gmail.com My friends, love is better than anger. Hope is better than fear. Optimism is better than despair. So let us be loving, hopeful and optimistic. And we'll change the world. - Jack Layton

3 2

PyPy2.7-v5.6.0 has been released
by Matti Picus 12 Nov '16

12 Nov '16

PyPy2.7 v5.6 released - stdlib 2.7.12 support, C-API improvements, and more https://morepypy.blogspot.com/2016/11/pypy27-v56-released-stdlib-2712-suppo… Thanks to all involved, please help get the word out. Success stories would be nice, we can help you do a guest blog post or link to your own posts. Matti

1 0

Re: [pypy-dev] PGO Optimized Binary
by Wang, Peter Xihong 11 Nov '16

11 Nov '16

Hi Maciej, This PGO is done for the interpreted code not the JITed code. Yash would give more details on this. Part of reason prompted us to do this work is due to 80% of the CPU cycles being spent in interpreted mode, while running OpenStack Swift in a real world scaled cluster setup. We've got 2% gain from this PGO effort on OpenStack, which is not bad at all. Thanks, Peter -----Original Message----- From: pypy-dev [mailto:pypy-dev-bounces+peter.xihong.wang=intel.com@python.org] On Behalf Of pypy-dev-request(a)python.org Sent: Friday, November 11, 2016 9:00 AM To: pypy-dev(a)python.org Subject: pypy-dev Digest, Vol 67, Issue 9 Send pypy-dev mailing list submissions to pypy-dev(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/pypy-dev or, via email, send a message with subject or body 'help' to pypy-dev-request(a)python.org You can reach the person managing the list at pypy-dev-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of pypy-dev digest..." Today's Topics: 1. Re: PGO Optimized Binary (Maciej Fijalkowski) 2. speed.pypy.org and old versions (Matti Picus) 3. release tarballs of the latest PyPy2.7 available on bitbucket (Matti Picus) 4. Re: speed.pypy.org and old versions (Carl Friedrich Bolz) ---------------------------------------------------------------------- Message: 1 Date: Thu, 10 Nov 2016 20:42:37 +0200 From: Maciej Fijalkowski <fijall(a)gmail.com> To: "Singh, Yashwardhan" <yashwardhan.singh(a)intel.com> Cc: Armin Rigo <armin.rigo(a)gmail.com>, "pypy-dev(a)python.org" <pypy-dev(a)python.org> Subject: Re: [pypy-dev] PGO Optimized Binary Message-ID: <CAK5idxTd841A3r=Q6ZjzF8e5AGYqrzMa8Oyqwep3uF0j4zA1qQ(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Hi 8% of that is very good if you can reproduce it across multiple runs (there is a pretty high variance I would think). You can also try running with --jit off. This gives you an indication of the speed of interpreter, which is a part of warmup On Wed, Nov 9, 2016 at 12:30 AM, Singh, Yashwardhan <yashwardhan.singh(a)intel.com> wrote: > Hi Armin, > > > Thanks for your feedback. > We ran one of the program suggested by you as an example for evaluation: > cd rpython/jit/tl > non-pgo-pypy ../../bin/rpython -O2 --source targettlr pgo-pypy > ../../bin/rpython -O2 --source targettlr > > We got the following results : > Non-Pgo pypy - > [Timer] Timings: > [Timer] annotate --- 7.5 s > [Timer] rtype_lltype --- 5.8 s > [Timer] backendopt_lltype --- 3.6 s > [Timer] stackcheckinsertion_lltype --- 0.1 s > [Timer] database_c --- 19.6 s > [Timer] source_c --- 2.6 s > [Timer] ========================================= > [Timer] Total: --- 39.2 s > > PGO-pypy : > [Timer] Timings: > [Timer] annotate --- 7.6 s > [Timer] rtype_lltype --- 5.1 s > [Timer] backendopt_lltype --- 3.1 s > [Timer] stackcheckinsertion_lltype --- 0.0 s > [Timer] database_c --- 18.5 s > [Timer] source_c --- 2.3 s > [Timer] ========================================= > [Timer] Total: --- 36.6 s > > The delta in performance between these two is about 8%. > > We are working on getting the data to identify the % of interpreted code vs the jited code for both the binaries. We are also working on creating a pull request to get a better feedback on the change. > > Regards > Yash > > ________________________________________ > From: Armin Rigo [armin.rigo(a)gmail.com] > Sent: Wednesday, November 02, 2016 2:18 AM > To: Singh, Yashwardhan > Cc: pypy-dev(a)python.org > Subject: Re: [pypy-dev] PGO Optimized Binary > > Hi, > > On 31 October 2016 at 22:28, Singh, Yashwardhan > <yashwardhan.singh(a)intel.com> wrote: >> We applied compiler assisted optimization technique called PGO or Profile Guided Optimization while building PyPy, and found performance got improved by up to 22.4% on the Grand Unified Python Benchmark (GUPB) from ?hg clone https://hg.python.org/benchmarks?. The below result table shows majority of 51 micros got performance boost with 8 got performance regression. > > The kind of performance improvement you are measuring involves only > short- or very short-running programs. A few years ago we'd have > shrugged it off as irrelevant---"please modify the benchmarks so that > they run for at least 10 seconds, more if they are larger"---because > the JIT compiler doesn't have a chance to warm up. But we'd also have > shrugged off your whole attempt---"PGO optimization cannot change > anything to the speed of JIT-produced machine code". > > Nowadays we tend to look more seriously at the cold or warming-up > performance too, or at least we know that we should look there. There > are (stalled) plans of setting up a second benchmark suite for PyPy > which focuses on this. > > You can get an estimate of whether you're looking at cold or hot code: > compare the timings with CPython. Also, you can set the environment > variable ``PYPYLOG=jit-summary:-`` and look at the first 2 lines to > see how much time was spent warming up the JIT (or attempting to). > > Note that we did enable PGO long ago, with modest benefits. We gave > up when our JIT compiler became good enough. Maybe now is the time to > try again (and also, PGO itself might have improved in the meantime). > >> We?d like to get some input on how to contribute our optimization recipe to the PyPy dev tree, perhaps by creating an item to the PyPy issue tracker? > > The best would be to create a pull request so that we can look at your > changes more easily. > >> In addition, we would also appreciate any other benchmark or real world use based workload as alternatives to evaluate this. > > You can take any Python program that runs either very shortly or not > faster than CPython. For a larger example (with Python 2.7): > > cd rpython/jit/tl > python ../../bin/rpython -O2 --source targettlr # 24 secs > pypy ../../bin/rpython -O2 --source targettlr # 39 secs > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev ------------------------------ Message: 2 Date: Fri, 11 Nov 2016 11:56:24 +0200 From: Matti Picus <matti.picus(a)gmail.com> To: "pypy-dev(a)python.org" <pypy-dev(a)python.org> Subject: [pypy-dev] speed.pypy.org and old versions Message-ID: <b2ea7e6e-9f7b-0d0f-0fa8-7bcdfbf13a97(a)gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed The landing page for speed.pypy.org, and the pull-downs on the other pages, are clogged with old results from previous versions. It seems the comparison page http://speed.pypy.org/comparison cannot even pull up the database info, and times out, making it useless. A quick fix could be to remove tags for much of the historical data, say for things more than 3 years old, keeping a smattering of significant versions? What you all think? Matti ------------------------------ Message: 3 Date: Fri, 11 Nov 2016 12:46:19 +0200 From: Matti Picus <matti.picus(a)gmail.com> To: "pypy-dev(a)python.org" <pypy-dev(a)python.org> Subject: [pypy-dev] release tarballs of the latest PyPy2.7 available on bitbucket Message-ID: <a36ca1db-1f7a-7ee4-0c66-78b3a4b5f67c(a)gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed https://bitbucket.org/pypy/pypy/downloads Please download and try them out, if there are no problems we will try for a Sunday release Matti ------------------------------ Message: 4 Date: Fri, 11 Nov 2016 14:34:30 +0100 From: Carl Friedrich Bolz <cfbolz(a)gmx.de> To: Matti Picus <matti.picus(a)gmail.com>, "pypy-dev(a)python.org" <pypy-dev(a)python.org> Subject: Re: [pypy-dev] speed.pypy.org and old versions Message-ID: <60f1f4dc-d085-42a6-b7f4-75579d03da55(a)email.android.com> Content-Type: text/plain; charset="utf-8" Hi Matti, Sounds good to me! Maybe we can keep the tag info around somewhere else (a file in extradoc?) in case we ever need it again. Thanks for doing this! Carl Friedrich On November 11, 2016 10:56:24 AM GMT+01:00, Matti Picus <matti.picus(a)gmail.com> wrote: >The landing page for speed.pypy.org, and the pull-downs on the other >pages, are clogged with old results from previous versions. >It seems the comparison page http://speed.pypy.org/comparison cannot >even pull up the database info, and times out, making it useless. >A quick fix could be to remove tags for much of the historical data, >say for things more than 3 years old, keeping a smattering of >significant versions? >What you all think? >Matti >_______________________________________________ >pypy-dev mailing list >pypy-dev(a)python.org >https://mail.python.org/mailman/listinfo/pypy-dev

1 0

pypy-dev November 2016