Mailman 3 March 2014 - pypy-dev

win32 and external function calls
by Matti Picus 22 Mar '14

22 Mar '14

1 0

How to check, that jit is turned on?
by KoDer 21 Mar '14

21 Mar '14

Hi all, I have a quite simple python script, which mostly don't use python dynamic features, but it almost two times slower unded latest pypy than under python 2.6.7. Is there any way to check, that jit actually working? pypy is builded from source with > python ../../rpython/bin/rpython -Ojit targetpypystandalone [PyPy 2.2.1 with GCC 4.8.1] on linux2 Thanks -- K.Danilov aka koder Skype:koder.ua Tel:+38-050-4030512

2 3

GSOC: Introduction and Interested in Numpy Improvements Project
by Rajul Srivastava 19 Mar '14

19 Mar '14

Hi all, My name in Rajul, and I am a final year undergraduate student at the Indian Institute of Technology Kharagpur. I wish to participate in Google Summer of Code 2014, and while going through the list of organisations, I came across PyPy. I am proficient with programming languages C/C++, Python, Java, Groovy, Ruby. I am very interested in the fields of Algorithms, Computational Sciences, and Software Engineering. I have always been interested in programming and in the past I have participated in Google Summer of Code 2012, with the organisation Network Time Foundation,working on the project "improving the Logging/Debugging System of Network Time Protocol Software". I have also interned in the Global Technology division of Barclays, during the summers of 2013, working with the Market Risk IT team. Besides I have worked on a few Research projects in the fields of Computational Finance, Complex Networks, and Computational Chemistry. I am currently working on my Thesis project in the field of Computational Sciences on a project titled "Network Analysis of Chemical Reactions". I have had courses in the fields of Programming and Data Structures, Complex Networks, Distributed Systems, Algorithms, Operations Research in the past. I have gone through the list of project ideas and I found all of the project ideas very interesting. Although I find all the projects listed worth a while, I am particularly interested in the "Numpy Improvements" project. I suppose that my programming background is suitable for these projects. I shall be grateful if anyone can help me and give me reference to the literature that I may use and also shed some light on how I can go about making a successful proposal. Thanks!! Best Regards, Rajul

2 1

Rollback interpreter state to fork for unittesting
by anatoly techtonik 19 Mar '14

19 Mar '14

Hi, I wonder if it possible to discard run-time changes to interpreter state and get back to some point in the past? One of the applications is forking to speed up unit tests - for it after interpreter is initialized and loaded unittest imports. -- anatoly t.

2 6

ssl and win32
by matti picus 19 Mar '14

19 Mar '14

I have been slowly trying to work on the failure on win32 of ssl, as reported in issue 1696 [0]. The issue now includes a 10 line test, using the stdlib test_ftplib classes. i tried converting the test to run on untranslated python, but am running into a very strange failure mode similar to the own-testing failure of ssl_wrap [1]. For some reason a failing call to socket functions is setting errno==2, which is not a valid socket call error [3] I would love to get some ideas about how to progress, or even better that someone else fix this. Matti [0] https://bugs.pypy.org/issue1696 [1] http://buildbot.pypy.org/summary/longrepr?testname=AppTestSSL.%28%29.test_s… [3] http://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).a…

1 0

Pypy garbage collection
by Martin Koch 18 Mar '14

18 Mar '14

Hi List I'm running a server (written in python, executed with pypy) that holds a large graph (55GB, millions of nodes and edges) in memory and responds to queries by traversing the graph.The graph is mutated a few times a second, and there are hundreds of read-only requests a second. My problem is that I no control over garbage collection. Thus, a major GC might kick in while serving a query, and with this amount of data, the GC takes around 2 minutes. I have tried mitigating this by guessing when a GC might be due, and proactively starting the garbage collector while not serving a request (this is ok, as duplicate servers will respond to requests while this one is collecting). What I would really like is to be able to disable garbage collection for the old generation. This is because the graph is fairly static, and I can live with leaking memory from the relatively few and small mutations that occur. Any queries are only likely to generate objects in the new generation, and it is fine to collect these. Also, by design, the process is periodically restarted in order to re-synchronize it with an authoritative source (thus rebuilding the graph from scratch), so slight leakage is not an issue here. I have tried experimenting with setting environment<http://pypy.readthedocs.org/en/latest/gc_info.html>variables as well as the 'gc' module, but nothing seems to give me what I want. If disabling gc for certain generations is not possible, it would be nice to be able to get a hint when a major collection is about to occur, so I can stop serving requests. I'm using the following pypy version: Python 2.7.3 (2.2.1+dfsg-1, Jan 24 2014, 10:12:37) [PyPy 2.2.1 with GCC 4.6.3] on linux2 An additional question: pypy 2.2.1 should have incremental GC; shouldn't that avoid long pauses due to garbage collection? Thanks, /Martin Koch - Senior Systems Architect - issuu.com

5 29

Re: [pypy-dev] RinohType and PyPy2
by Maciej Fijalkowski 11 Mar '14

11 Mar '14

Hi Brecht. I must say I've been trying to understand what's going on and I'm failing so far. Thanks for a valuable benchmark! And yes, we're working on improving the warmup time (ETA unknown though) On Sun, Mar 2, 2014 at 12:11 PM, Brecht Machiels <brecht(a)mos6581.org> wrote: > Thanks Maciej, > > sys._getframe was introduced by "magicsuper", which I used to avoid refactoring all super() calls. I've done that now and there shouldn't be any more sys._getframe calls. You can pull in this commit from the pypy2 branch. > > Unfortunately, this didn't improve performance much. PyPy now takes 26 seconds. Any other ideas? > > Best regards, > Brecht > > ---- On Sun, 02 Mar 2014 08:19:35 +0100 Maciej Fijalkowski wrote ---- > >>the first obvious thing that jumps at me is your casual use of >>sys._getframe - the JIT aborts in this case and proceeds to the >>interpreter (so you pay the price for JITting, while you also pay the >>prace for not having compiled assembler). That probably does not >>explain everything, but please don't use sys._getframe in production >>code if you want the JIT to be fast. >> >>On Sun, Mar 2, 2014 at 12:34 AM, Brecht Machiels > wrote: >>> Hello, >>> >>> I've managed to backport RinohType to Python 2 (took me only a couple of hours thankfully). >>> >>> Results on my Celeron T3000 (Arch Linux x86_64): >>> CPython 3.3.4 14 s >>> PyPy3 2.1.0-beta1 61 s >>> CPython 2.7.6 15 s >>> PyPy 2.2.1 35 s >>> >>> If you want to give it a try (no external dependencies): >>> >>> git clone --branch pypy2 https://github.com/brechtm/rinohtype.git >>> cd rinohtype/examples/rfic2009 >>> rm -rf template.ptc; PYTHONPATH=../.. pypy template.py >>> >>> >>> While PyPy2 performs better than PyPy3, it's still much slower than CPython. Is RinohType hitting a weak spot in PyPy? Any hints on what I can do to improve performance? >>> >>> Best regards, >>> Brecht >>> >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev(a)python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >> > > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev

3 2

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

slow-ish multithreaded primitives
by Dima Tisnek 10 Mar '14

10 Mar '14

Hey all, I wrote some threaded code and ran it under different version of python, and unexpectedly high overhead is observed: cpython 2.6.8. cpython 2.7.6, pypy 2.2.1: 14~53ms cpython 3.3.4: pypy 2.1.0 beta py3 mode: <1ms is someone here interested in getting to the bottom of it? or getting pypy py2 mode in line with py3 mode? the set up: 1 lock 2 threads each thread has own condition variable both condition variables share the lock mentioned above. threads wake each other up in turn. arch x86_64, linux 3.12.9, up to date libc/rt/pthread sorry I don't have code handy, it's part of a larger project, but if someone's interested, please reply and I'll hack up a short test case. thanks, d.

4 10

win32 failures on own tests
by Matti Picus 10 Mar '14

10 Mar '14

2 1