Mailman 3 July 2011 - pypy-dev

Building pypy on freebsd 64bit
by Shadrin Eugene 18 Nov '11

18 Nov '11

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene

5 7

Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
by Ian Ozsvald 16 Nov '11

16 Nov '11

Dear all, I've published v0.2 of my High Performance Python tutorial write-up from the session I ran at EuroPython: http://ianozsvald.com/2011/07/25/high-performance-python-tutorial-v0-2-from… Antonio - you asked earlier if the 'expanded math' version of the Mandelbrot solver (using doubles rather than complex numbers) would be faster - I've timed it and it is a bit faster with a nightly build of PyPy, but nowhere near as fast at ShedSkin's generated C output (details below). Maciej - thanks for pointing me at the numpy module. I've added a tiny section showing numpy in PyPy but I haven't converted the Mandelbrot solver to use it (even finishing v0.2 took longer than I'd thought). I'm hoping that some more exposure in the report might bring in more volunteers from outside. Here's a clip from the report in the PyPy section: "By running pypy pure_python.py 1000 1000 on my MacBook it takes 5.9 seconds, running pypy pure_python_2.py 1000 1000 takes 4.9 seconds. (Ian - the only difference with pure_python_2.py is that local dereferences in the tight loop are moved outside the loop, causing fewer dereference operations) As an additional test (not shown in the graphs) I ran pypy shedskin2.py 1000 1000 which runs the expanded math version of the shedskin variant below (this replaces complex numbers with floats and expands abs to avoid the square root). The shedskin2.py result takes 3.2 seconds (which is still much slower than the 0.4s version compiled using shedskin)." The pure_python src is here: https://github.com/ianozsvald/EuroPython2011_HighPerformanceComputing/tree/… shedskin2.py is available here: https://github.com/ianozsvald/EuroPython2011_HighPerformanceComputing/tree/… I haven't tested whether the warm-up periods for PyPy are significant, possibly they account for much of the difference between ShedSkin and PyPy? I want to revisit this but for the next few weeks I have to go back to other projects. I hope the report brings in some new folk for PyPy, Ian. -- Ian Ozsvald (A.I. researcher, screencaster) ian(a)IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

5 8

Re: [pypy-dev] bounties for pypy
by Laura Creighton 16 Sep '11

16 Sep '11

>The idea was also to possibly attract new developers ... for example, if >there would be "10 days in money" for adapting py2exe, I am sure many wou >ld >jump to solve this puzzle. This is sort of a bad example. Because py2exe embeds CPython, and we wouldn't want to do that. So what we would probably want to do is to make some general tool that willmake a windows binary, or a mac one, and get rid of the need for bzfreeze and friends. So now you are looking at a general embedding solution, and that is more than 10 days worth of work. But I get the idea. <snip> >> my dream was of a trustee service: after somebody commits to do the wor >k, >the pledgers have to pay to a trustee. then the work is done. then the >trustee pays the worker. This is one of the things I want to talk with fundedbyme about. But having an explicit trustee is a new idea. I think the pypy core developers are already rather well trusted in this community, but this may be important to new developers who aren't as well known. And it handles the problem' of 'I got sick and cannot do this any more' more gracefully than other solutions. >Hmmm.... a structure could be: > >- service provider does the technical stuff, as in: > # website > # collect pledges > # handle project description > # collect money > # distribute money after feature completion fundedbyme has sort of indicated an interst in doing this (except they were talking about distribution before, and I was leaving project description to the project, not outsiders). I will follow up on this when I get back home to Sweden. >- PSF / pypy-foundation / whateverfoundation provides the trust > >Thanks for confirming the need for such a thing! > >Harald Thanks once again for seeing a marketing solution that nerds like us often miss. Laura

6 12

Great results for MyHDL simulations
by Jan Decaluwe 01 Sep '11

01 Sep '11

I am seeing great improvements for MyHDL simulations by using PyPy, and I have written a page about it: http://www.myhdl.org/doku.php/performance I have asked for feedback from the MyHDL community to confirm these results, before creating some more buzz in the hardware design world :-) Thanks! Jan Decaluwe -- Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com Python as a HDL: http://www.myhdl.org VHDL development, the modern way: http://www.sigasi.com Analog design automation: http://www.mephisto-da.com World-class digital design: http://www.easics.com

3 4

work on libpypy.so
by Roberto De Ioris 20 Aug '11

20 Aug '11

Hi everyone, my company has a bunch of C apps (mainly the uWSGI application server container and a scriptable DNS server) embedding CPython in their core. We would like to start using PyPy, so our need is having a workable libpypy.so implementation (we do not need it to be compatible with CPython api, we can rewrite those parts, expecially in uWSGI, without problems). I have searched for some work already done, but it looks like no one is pushing it. I apply myself for working on this area (if no-one is already doing it). So if there are no objections, i would start posting my proposals (and code) in the next few days. -- Roberto De Ioris http://unbit.it

5 5

newbie needs pypy setup help
by Tom Roche 10 Aug '11

10 Aug '11

(Apologies if - this is not the appropriate venue for this question, but I don't see any other pypy lists, and I'm finding the other docs a bit inscrutable. - I sound too inexperienced or unpythonic: I've spent most of my coding life in java and perl. ) Please advise how to configure pypy to run other python code. Why I ask: I've started running a model implemented in python. Unfortunately a run on "normal" python 2.6.x or 2.7.x requires - 130 min on my ubuntu laptop (on which working would be more convenient) - 55 min on the best build machine to which I currently have access However I have read that this model runs 5x faster under pypy, so I wanna get me that! Unfortunately my current ubuntu me@it:~$ lsb_release -ds > Ubuntu 10.04.3 LTS # yes, I am planning to upgrade Real Soon Now me@it:~$ uname -rv > 2.6.32-33-generic #70-Ubuntu SMP Thu Jul 7 21:13:52 UTC 2011 is too down-level to install the available pypy RPM, so I instead did URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux64.tar.bz2" TMP_DIR_ROOT="/tmp/pypy" for CMD in \ "rm -fr ${TMP_DIR_ROOT}" \ "mkdir -p ${TMP_DIR_ROOT}" \ "pushd ${TMP_DIR_ROOT}" \ "wget -O - ${URI} | tar xvjf -" \ "ls -alh" \ "popd" \ ; do echo -e "${CMD}" eval "${CMD}" done # check path names, then TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux64" TARGET_DIR_ROOT="/opt/pypy-c-jit-1.5.0-alpha0" TARGET_PYPY_EXEC="${TARGET_DIR_ROOT}/bin/pypy" USR_PYPY_EXEC="/usr/local/bin/pypy" for CMD in \ "sudo mkdir -p ${TARGET_DIR_ROOT}" \ "sudo cp -r ${TMP_DIR_ROOT}/* ${TARGET_DIR_ROOT}/" \ "sudo chmod 755 ${TARGET_DIR_ROOT}/" \ "sudo chmod 755 ${TARGET_DIR_ROOT}/bin/" \ "sudo chmod 755 ${TARGET_PYPY_EXEC}" \ "sudo ls -al ${TARGET_PYPY_EXEC}" \ "sudo ln -s ${TARGET_PYPY_EXEC} ${USR_PYPY_EXEC}" \ "sudo ls -al ${USR_PYPY_EXEC}" \ "${USR_PYPY_EXEC} --version" \ "which pypy" \ "pypy --version" \ ; do echo -e "${CMD}" eval "${CMD}" done The good news is, I can now me@it:~$ ls -al $(which pypy) > lrwxrwxrwx 1 root root 37 2011-07-30 16:06 /usr/local/bin/pypy -> /opt/pypy-c-jit-1.5.0-alpha0/bin/pypy me@it:~$ pypy --version > Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34) > [PyPy 1.5.0-alpha0 with GCC 4.4.3] But when I try to actually *run* the @#$%^&! thing, it spews: me@it:~$ pypy > debug: WARNING: library path not found, using compiled-in sys.path and sys.prefix will be unset > 'import site' failed > Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34) > [PyPy 1.5.0-alpha0 with GCC 4.4.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > debug: OperationError: > debug: operror-type: ImportError > debug: operror-value: No module named _pypy_interact me@it:~$ pypy -c 'import sys; print sys.path' > debug: WARNING: library path not found, using compiled-in sys.path and sys.prefix will be unset > 'import site' failed > ['', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib_pypy', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib-python/modified-2.7', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib-python/2.7', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib-python/modified-2.7/lib-tk', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib-python/2.7/lib-tk', '/home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/lib-python/2.7/plat-linux2'] What do I need to do to fix its library path? Is this where translation is required? (Again, please note that I *have read* http://codespeak.net/pypy/dist/pypy/doc/getting-started-python.html but I'm finding difficulty understanding what one must do to *use* pypy, vs what one must do to *hack* pypy, which is not my usecase.) TIA, Tom Roche <Tom_Roche(a)pobox.com>

8 31

PyPy at last infinitely fast
by Miquel Torres 07 Aug '11

07 Aug '11

Hi, speed.pypy.org currently shows a very encouraging performance picture for PyPy: it is "infinite times faster than CPython". No, it is note yet April 1st. Codespeed creates the front page plots using the latest tested revision... which currently has no data for pypy-c-jit (32 bits). There is already a ticket for that issue, as well as to fix the Changes view letting you choose a revision with no data. The assumption was that a revision would be tested for all executables of a given project, which is no longer the case for PyPy. Still, even though there will be a fix, it may be a good idea to test all exes for the same revision to easy comparisons. Cheers, Miquel

3 20

pypy for debian?
by Tom Roche 31 Jul '11

31 Jul '11

Anyone packaging pypy for debian? Why I ask: I'm putting a FOSS project on ShiningPanda, which runs Debian 6 (squeeze), for CI. After seeing the speedup from pypy on my own hardware, I'd like to use pypy on SP, but they don't currently offer that option. Hence I'd like to ask SP to provide pypy along with the CPython versions they provide (rather than hafta provide it myself in each build environment I use, which would add to my setup time, defeating the purpose), but before I did that, I'd like to get some idea of how much work would be entailed, i.e., how much pain I'd be asking them to endure. Adopting pypy would presumably be trivial for SP if there were pypy .deb, or even a PPA, but I'm not seeing anything like that after ubuntu hardy. Am I missing something? I see RH (appears to be) regularly packaging pypy, so I'm hoping someone's doing this for debian. If not, there's always `alien`. TIA, Tom Roche <Tom_Roche(a)pobox.com>

2 1

Re: [pypy-dev] newbie needs pypy setup help
by Tom Roche 31 Jul '11

31 Jul '11

Ken Watford Sat, 30 Jul 2011 21:18:23 -0400 > sudo chmod -R go+rX /opt/pypy-c-jit-1.5.0-alpha0 That's it! So the tarball install that works (for me) is URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux64.tar.bz2" TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux64" TARGET_DIR_ROOT="/opt/pypy-c-jit-1.5.0-alpha0" TARGET_PYPY_EXEC="${TARGET_DIR_ROOT}/bin/pypy" USR_PYPY_EXEC="/usr/local/bin/pypy" for CMD in \ "rm -fr $(dirname ${TMP_DIR_ROOT})" \ "mkdir -p ${TMP_DIR_ROOT}" \ "pushd $(dirname ${TMP_DIR_ROOT})" \ "wget -O - ${URI} | tar xvjf -" \ "ls -alh" \ "popd" \ "sudo rm -fr ${TARGET_DIR_ROOT}" \ "sudo mkdir -p ${TARGET_DIR_ROOT}" \ "sudo cp -r ${TMP_DIR_ROOT}/* ${TARGET_DIR_ROOT}/" \ "sudo chmod -R go+rX ${TARGET_DIR_ROOT}/" \ "sudo ln -s ${TARGET_PYPY_EXEC} ${USR_PYPY_EXEC}" \ "which pypy" \ "sudo ls -al ${USR_PYPY_EXEC}" \ "${USR_PYPY_EXEC} --version" \ "sudo ls -al ${TARGET_DIR_ROOT}" \ "pypy --version" \ "pypy" \ "popd" \ ; do echo -e "$ ${CMD}" eval "${CMD}" done which tails > $ pypy > Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34) > [PyPy 1.5.0-alpha0 with GCC 4.4.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > And now for something completely different: ``the future has just begun'' > >>>> Thanks! Tom Roche <Tom_Roche(a)pobox.com>

1 0

pypy array module memory leak?
by Berend De Schouwer 30 Jul '11

30 Jul '11

The following program has constant (+- 10 MB) memory usage in CPython, but it quickly leaks massive amounts of memory in pypy. It simply assigns one bit of an array to another slice. The length of the array remains constant (so there's nothing to delete from inside the program) pypy-1.5 (pypy-c-jit-43780-b590cf6de419-linux) import array import time a = array.array('i', 'a' * 100000) for i in range(1,100000): a[1:1000] = a[2001:3000] time.sleep(0.0001) # So you can go check mem usage. I apologise for the disclaimer The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.

3 5