Mailman 3 August 2015 - pypy-dev

Poor performance for Krakatau
by Robert Grosse 10 Aug '15

10 Aug '15

In some cases, Pypy performs worse than CPython when running Krakatau, and it has gotten a lot worse from 2.3 to 2.5 My benchmark is https://github.com/Storyyeller/Krakatau/tree/pypy_benchmark (for future reference, commit d889c7f44723e6d66a3630681f0385f173317dc9) You can run it with pypy Krakatau\benchmark.py -path <path to Java rt.jar> I get the following timings CPython: 17.43s Pypy 2.3: 38.48s (build pypy-c-jit-71056-c8e3b8cbc843-win32) Pypy 2.4: 39.92s (build pypy-c-jit-72200-375133966c12-win32) Pypy 2.5: 49.94s (build pypy-c-jit-74404-9fd586fe0fe5-win32) Not only was it slower than CPython to begin with, but it got much worse recently.

3 4

serialize and deserialize objects in Pypy
by Yicong Huang 07 Aug '15

07 Aug '15

Hi, Are there any good methods to serialize Pypy object to C++ char* and deserialize to Pypy object from C++ char*? Our scenario like this: We have at least two C++ process running, and each process embeded Pypy to execute some functions. And we woule like to reuse Pypy object from one process to another process. For primitive objects, int/char/bool/double etc., we could use cffi to pass objects. Pypy object -> c++ process 1 -> c++ process 2 -> Pypy object But for complex objects, e.g. list/array/dict/tuple, are there any methods?

4 8

Running pypy unit tests
by Vaibhav Sood 06 Aug '15

06 Aug '15

Hi All, I work on Python through which I came across pypy and I am interested in making some open-source contributions to the project. As first steps I tried building the source code (successfully) and running the unit tests as per the "Getting Started Developing with PyPy" page but I run into an issue The way I run the tests is "./pytest.py <folder>" for each folder in the source code, I get the test results all folders but the command hangs for one particular folder "pypy/module" For example, ./pytest.py -rfExs pypy/module >> Running this never returns, doing a ps -ef I see multiple pytest.py processes running I run the tests on a x86 VM with 4GB RAM. I am not sure if I am doing this right but is there a way around this issue? Also is there any recommended way to run all the pypy tests and is there any other recommended strategy to test new code added to pypy that I am missing? I apologize for the slightly lengthy mail and the multiple questions Thanks! Vaibhav DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

3 3

Re: [pypy-dev] Running pypy unit tests
by Matej Stuchlik 06 Aug '15

06 Aug '15

Hi Armin, ----- Original Message ----- > From: "Armin Rigo" <arigo(a)tunes.org> > To: "Vaibhav Sood" <vaibhav_sood(a)persistent.com> > Cc: pypy-dev(a)python.org > Sent: Thursday, August 6, 2015 10:46:27 AM > Subject: Re: [pypy-dev] Running pypy unit tests > > Hi Vaibhav, > > On 5 August 2015 at 10:22, Vaibhav Sood <vaibhav_sood(a)persistent.com> wrote: > > The way I run the tests is “./pytest.py <folder>” for each folder in the > > source code, I get the test results all folders but the command hangs for > > one particular folder “pypy/module” > > By "each folder" we mean "each individual 'test' subfolder". The > pypy/module directory contains a large number of subdirectories, each > with their own tests, so you should not run all the pypy/module tests > at once. > > Yes, there is an automatic way. However in general we don't use it > ourselves. It is used automatically by the nightly test runs which > show up at http://buildbot.pypy.org (when the web site is not down > like now...). Could you please describe the automatic way in a little more detail? buildbot.pypy.org seems to be down right now, so I can't look it up there. We run all the tests when building pypy for Fedora and I'm pretty sure the way we run them is not exactly current by now. :) Matt > Instead, when we do changes somewhere, we run directly the tests in > the "test" subdirectory of where we did the changes, and if they pass, > we commit. In the rare case where this breaks things at some > unrelated place, we will notice the next morning after the nightly > run, and fix it then. > > > A bientôt, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev >

2 1

Re: [pypy-dev] RinohType and PyPy2
by Maciej Fijalkowski 05 Aug '15

05 Aug '15

Hi Brecht We tried the strbuf with rinohtype and (after warmup) it's the speed of cpython, which is bad. investigating some more.... On Sun, Mar 2, 2014 at 12:34 AM, Brecht Machiels <brecht(a)mos6581.org> wrote: > Hello, > > I've managed to backport RinohType to Python 2 (took me only a couple of hours thankfully). > > Results on my Celeron T3000 (Arch Linux x86_64): > CPython 3.3.4 14 s > PyPy3 2.1.0-beta1 61 s > CPython 2.7.6 15 s > PyPy 2.2.1 35 s > > If you want to give it a try (no external dependencies): > > git clone --branch pypy2 https://github.com/brechtm/rinohtype.git > cd rinohtype/examples/rfic2009 > rm -rf template.ptc; PYTHONPATH=../.. pypy template.py > > > While PyPy2 performs better than PyPy3, it's still much slower than CPython. Is RinohType hitting a weak spot in PyPy? Any hints on what I can do to improve performance? > > Best regards, > Brecht > > > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev

2 2

Building on case-insensitive file system
by Cyrus Daboo 05 Aug '15

05 Aug '15

Hi, OS X defaults to having a case-insensitive file system. I believe there is a problem with building pypy from source on such a system. During the build process I see files being written to the jitcodes directory in the /tmp build directory. There are some "W_ArrayTypeX.*" files being written there, where "X" is a typecode (which I believe comes from pypy/module/array/interp_array.py). The allowable codes come from the set "c, b, B, u, h, H, i, I, l, L, f or d". Note that there are a mixture of lower and upper case characters that are the same. When I build on OS X, with an option to force an error when trying to create a file with a name that matches another, but with different case, I see this: [translation:ERROR] EEXIST: [File exists]: file('/tmp/usession-release-2.6.x-0/jitcodes/W_ArrayTypel._charbuf_stop', 'w') That is because "W_ArrayTypeL._charbuf_stop" exists - i.e., "l" and "L" are in conflict. Now, I am not a PyPy dev expert, so I have no idea what the impact of this might be, but it seems wrong to me. Though I cannot detect any obvious problems in the built pypy binary. I am not sure what an appropriate fix for this would be. Perhaps changing the typecode set to use unique characters (case insensitive) for all the different types? -- Cyrus Daboo

1 0

Cobra down
by Maciej Fijalkowski 05 Aug '15

05 Aug '15

According to David, cobra/buildbot will be down for a day or two

1 0

For Donations, List How Much is Left (or Spent)?
by Donald Stufft 03 Aug '15

03 Aug '15

Hey! So out of someone complaining about PyPy's lack of an up to date port of Python 3, it struck me that for someone just now looking into PyPy it looks like PyPy is sitting on almost 60k USD in order to port PyPy3 from 3.2 to 3.4. However the reality (via fijall) is that most of it, roughly 50k has already been spent going from nothing to 3.2. It might be a good idea to have the numbers reflect how much of the fundraised money has already been spent so that people get a more accurate reflection on just how much money there is left to take the task at hand from where it is now to where folks want it to be. My assumption is that people are going to feel a lot more like their money is going to be useful if it's not just adding onto the tail of what looks to be a fairly major pot of money and thus are more likely to actually donate. This of course doesn't just apply to Python 3 porting, the same thing is true for all of the fundraisers that PyPy is running. Without more information I have no way to evaluate if PyPy is sitting with cash reserves for that task and it's waiting for someone to work on it, or if it's mostly gone and it needs more funding to get any more effort applied to it. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

6 9

pypy-2.3.1 memory profiling strategies
by jim horng 03 Aug '15

03 Aug '15

Hi all, Not sure if it's appropriate to post here since there's no pypy-user group :) My question is as title, we got memory leak on pypy process and process will down when out of memory, only on production site. Our simplified environment as below: * OS: Centos 6 * pypy-2.3.1 objgraph is seems the only profiling library we can use in this env, and only with its partial function of printing all current objects in memory instead of any further info such as references (.getrefcount not implemented). It turns out we can only see lots of "int", "str", "list" objects seems leaking rather than knowing who are using them or whom they are using. :( Our constraint are * it's hard to change production python runtime since it might affect to our users * we cannot reproduce on other environment "pmap" produced data only shows memory growing in a [anon] block. Please advise if there's other tool/methodologies to attack this problem, thanks a lot in advance :) Best Regards, Jim(洪懷謙)

3 3

Best way to use py.test with RPython projects and travis-ci?
by Sarah Mount 03 Aug '15

03 Aug '15

This week someone pushed a nice commit to a Python repository I contribute to, which sped up Travis builds by upgrading pip and using the cache to store requirements. I thought "that's neat, I should try that trick in my own (R)Python projects". And that, friends, is how I entered a whole new world of pain and suffering. It seems that if you install py.test using pip and cache pip packages in Travis, like this: https://travis-ci.org/futurecore/revelation/builds/73584887 I think because PyPy packages its own version of py.test and the two get confused. On the other hand, if you strip py and py.test from PyPy then you can't translate your interpreter: https://travis-ci.org/futurecore/revelation/builds/73694483 What is the recommended way to manage this stuff? Thanks, Sarah -- Dr. Sarah Mount, Senior Lecturer, University of Wolverhampton website: http://www.snim2.org/ twitter: @snim2

2 1