Mailman 3 November 2010 - pypy-dev

Building pypy on freebsd 64bit
by Shadrin Eugene 18 Nov '11

18 Nov '11

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene

5 7

Женская логика
by Vovan 02 Dec '10

02 Dec '10

Хотите понимать женщин? Знать реальные их мысли, когда они пытаются спрятать их за совершенно отвлеченными речами? На практических тренингах Академии Знакомств вы в реальных условиях научитесь быть предупредительным и понимающим мужчиной для женщин. http://samec.org.ua/?p=202 Не верите? Попробуйте, вам понравится. Желаю вам больших научных успехов.

2 1

a possible leak in the object namespace...
by Alex A. Naanou 01 Dec '10

01 Dec '10

Hi All, Decided to test the 1.4 with a couple of my usecases, got quite surprising results... To keep it short, I have several usecases that involve replacing the object dictionary. Here's a simple example: ---cut--- class D(dict): def __setitem__(self, key, value): print 'updating attr "%s" to "%s"...' % (key, value) super(D, self).__setitem__(key, value) class X(object): pass x = X() x.__dict__ = D() x.a = 1 # will print a nice log string... # NOTE: this will not print anything in CPython (see P.S.) --uncut-- With the release of version 1.4, I decided to test these usecases out and benchmark them on PyPy and 15 minutes later I got results that were surprising to say the least... Expectations: 1) the normal/native namespace should have been a bit faster than the hooked object on the first run. Both cases should have leveled to about the same performance after the JIT finished it's job +/- a constant. 2) all times should have been near constant. What I got per point: 1) the object with native dict was slower by about three orders of magnitude than the object with a hooked namespace. 2) sequential write benchmark runs on the normal object did not level out, as they did with the hook, rather, they exhibited exponential times (!!) For details and actual test code see the attached file. P.S. This specific functionality is a weak point in CPython for two reasons: 1) writing to .__dict__ demands a subclass of dict (not a real problem, though over-restrictive in my view) 2) all interactions with the namespace are done directly via the Python C-API, completely neglecting the high-level dict interface. Thanks! -- Alex.

2 5

which xml libraries? was (Re: PyPy 1.4 released)
by René Dudfield 01 Dec '10

01 Dec '10

Hi, what xml libraries are people using with pypy? What is working well? cu, On Sun, Nov 28, 2010 at 9:48 AM, Maciej Fijalkowski <fijall(a)gmail.com>wrote: > Hey. > > On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar <phyo.arkarlwin(a)gmail.com> > wrote: > > i got python-magic working , after i installed without easy_install > > (easy_install fail because it tried to install ctypes). > > great > > > > > Now what is not working is python-lxml , which is very important for my > project. > > lxml won't work out of the box. if you think it's important enough, > you can try to port cython to generate something saner (right now what > it generates won't work on cpyext). > > Cheers, > fijal > > > > > here are the errors: > > > > Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir > > /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2 > > Building lxml version 2.3.beta1. > > NOTE: Trying to build without Cython, pre-generated > > 'src/lxml/lxml.etree.c' needs to be available. > > Using build configuration of libxslt 1.1.26 > > Building against libxml2/libxslt in the following directory: /usr/lib > > src/lxml/lxml.etree.c:75: error: conflicting types for ‘Py_buffer’ > > /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of > > ‘Py_buffer’ was here > > > > > > Had Anyone got lxml working in pypy successfully? > > > > On 11/27/10, Antonio Cuni <anto.cuni(a)gmail.com> wrote: > >> On 27/11/10 03:09, Phyo Arkar wrote: > >>> libmagic python fails to work on pypy (python-magic) > >>> > >>> it uses ctypes and easy-install try to download and instaill it , but > it > >>> fails. > >>> > >>> how to enable ctypes on pypy? > >> > >> Hi Phyo, > >> ctypes *is* enabled on pypy by default. > >> > >> If python-magic does not work, it can either: > >> > >> 1) be a pypy bug: please report it as an issue (possibly with a simple > >> failing > >> test and the full traceack) > >> > >> 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot > >> really be supported by pypy due to the internal differences with CPython > >> > >> ciao, > >> Anto > >> > > _______________________________________________ > > pypy-dev(a)codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > > _______________________________________________ > pypy-dev(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev >

6 14

PyPy 1.4 released
by Maciej Fijalkowski 30 Nov '10

30 Nov '10

=============================== PyPy 1.4: Ouroboros in practice =============================== We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough in our long journey, as PyPy 1.4 is the first PyPy release that can translate itself faster than CPython. Starting today, we are using PyPy more for our every-day development. So may you :) You can download it here: http://pypy.org/download.html What is PyPy ============ PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison) Among its new features, this release includes numerous performance improvements (which made fast self-hosting possible), a 64-bit JIT backend, as well as serious stabilization. As of now, we can consider the 32-bit and 64-bit linux versions of PyPy stable enough to run `in production`_. Numerous speed achievements are described on `our blog`_. Normalized speed charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_ are available on benchmark website. For the impatient: yes, we got a lot faster! More highlights =============== * PyPy's built-in Just-in-Time compiler is fully transparent and automatically generated; it now also has very reasonable memory requirements. The total memory used by a very complex and long-running process (translating PyPy itself) is within 1.5x to at most 2x the memory needed by CPython, for a speed-up of 2x. * More compact instances. All instances are as compact as if they had ``__slots__``. This can give programs a big gain in memory. (In the example of translation above, we already have carefully placed ``__slots__``, so there is no extra win.) * `Virtualenv support`_: now PyPy is fully compatible with virtualenv_: note that to use it, you need a recent version of virtualenv (>= 1.5). * Faster (and JITted) regular expressions - huge boost in speeding up the `re` module. * Other speed improvements, like JITted calls to functions like map(). .. _virtualenv: http://pypi.python.org/pypi/virtualenv .. _`Virtualenv support`: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html .. _`in production`: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software… .. _`our blog`: http://morepypy.blogspot.com .. _`pypy 1.4 and pypy 1.3`: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,… .. _`pypy 1.4 and cpython 2.6`: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,… Cheers, Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, Amaury Forgeot d'Arc, Armin Rigo and the PyPy team

7 11

PyPy-JIT memory usage compared to CPython?
by Andy 30 Nov '10

30 Nov '10

Hi, Congrats on the 1.4 release! >From the benchmark results on your site, the performance looks great. I was wondering if you have any data showing relative memory usage for the various benchmark. I'm particularly interested in the memory usage of PyPy-JIT vs. CPython when running Django. In general would PyPy-JIT increase or decrease memory usage? Thanks Andy

2 3

Re: [pypy-dev] gdbm
by Laura Creighton 29 Nov '10

29 Nov '10

In a message of Tue, 16 Nov 2010 02:22:56 +0100, "Amaury Forgeot d'Arc" writes: >Hi, > >2010/11/16 Dan Stromberg <drsalists(a)gmail.com>: >> I've put together a gdbm wrapper module using ctypes, and tested agains >t >> pypy with the included test_gdbm. Is there interest in adding it to pyp >y? >> >> It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone >wants >> to look it over. > >I don't know exactly what gdbm is, but it looks very similar to dbm.py >already in pypy: >http://codespeak.net/svn/pypy/trunk/lib_pypy/dbm.py Both gdbm and dbm are part of the python standard library, so I'd guess we'd better have both of them. http://docs.python.org/library/gdbm.html Laura >Amaury Forgeot d'Arc

3 8

double trouble with JIT
by wlavrijsen＠lbl.gov 29 Nov '10

29 Nov '10

Hi, with the Reflex branch, and the fast path enabled, I sometimes run into pbs with doubles in libffi. Sometimes the results are wrong, sometimes there's a crash, and sometimes there's: self = <C object ffi_cif (opaque) at 0xbfb8848> def _ctypes_storage_was_allocated(self): addr = ctypes.cast(self._storage, ctypes.c_void_p).value if addr in ALLOCATED: > raise Exception("internal ll2ctypes error - " "double conversion from lltype to ctypes?") E Exception: internal ll2ctypes error - double conversion from lltype to ctypes? Any ideas? Thanks, Wim -- WLavrijsen(a)lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net

2 5

test_all.py -A behavior
by Dan Roberts 29 Nov '10

29 Nov '10

I'm having trouble running the applevel tests on a translated pypy-c (without jit). I have this problem with recent trunk and my psycopg2 compatibility branch. Translating both with: python translate.py targetpypystandalone.py --withmod-cpyext and running tests with: translator/goal/pypy-c test_all.py -A both trunk and my branch have multiple errors on module/bz2/test/test_bz2_compdecomp.py tests, one failure in module/posix/test/test_posix2.py and both hang on the 8th test on module/signal/test/test_signal.py . This wouldn't be too weird if all three of those test suites run successfully via: translator/goal/pypy-c test_all.py -A $SUITE on both my branch and trunk. I'm on Ubuntu 10.04 32 bit. I tried again with trunk just now with the same behavior. Is this a test_all.py bug? Can anyone with a pypy-c confirm this behavior? Cheers, Dan

2 1

pypy 1.4 and virtualenv 1.5.1
by Michael Foord 26 Nov '10

26 Nov '10

Hello all, Congratulations on the new release. Unfortunately it seems that virtualenv 1.5.1 and pypy 1.4 do not play well together on Mac OS X. At least the "--distribute" option doesn't work, which is needed by tox. I tried to create an issue on the PyPy issue tracker, but new registrants are unable to create issues: Here is the output on Linux: $ virtualenv -p /compile/pypy-1.4/bin/pypy pypy --distribute --no-site-packages Running virtualenv with interpreter /compile/pypy-1.4/bin/pypy New pypy-c executable in pypy/bin/pypy Also creating executable in pypy/bin/pypy-c Installing distribute....................................................................................................................................................................................done. Here is the failed output on OS X: $ virtualenv -p /compile/pypy-1.4-osx/bin/pypy pypy --distribute --no-site-packages Running virtualenv with interpreter /compile/pypy-1.4-osx/bin/pypy New pypy-c executable in pypy/bin/pypy Also creating executable in pypy/bin/pypy-c Please make sure you remove any previous custom paths from your /Volumes/Second Drive/michael/.pydistutils.cfg file. Installing distribute................................done. Complete output from command /compile/pypy/bin/pypy /compile/pypy/bin/easy_install /Library/Frameworks/Python.fra...ar.gz: Traceback (most recent call last): File "app_main.py", line 33, in run_toplevel IOError: [Errno 2] No such file or directory: '/compile/pypy/bin/easy_install' ---------------------------------------- Traceback (most recent call last): File "app_main.py", line 33, in run_toplevel File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py", line 1647, in <module> main() File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py", line 558, in main prompt=options.prompt) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py", line 656, in create_environment install_pip(py_executable) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py", line 415, in install_pip filter_stdout=_filter_setup) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py", line 624, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /compile/pypy/bin/pypy /compile/pypy/bin/easy_install /Library/Frameworks/Python.fra...ar.gz failed with error code 1 The nice thing is that for unittest2 and mock all tests pass with pypy 1.4. Not unexpected, but still nice. All the best, Michael Foord -- http://www.voidspace.org.uk

2 2