Mailman 3 December 2010 - pypy-dev

Building pypy on freebsd 64bit
by Shadrin Eugene 18 Nov '11

18 Nov '11

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene

5 7

pypy GC on large objects Re: funding/popularity?
by René Dudfield 04 Jan '11

04 Jan '11

Hello, I think this is a case where the object returned by ctypes.create_string_buffer() could use a correct __sizeof__ method return value. If pypy supported that, then the GC's could support extensions, and 'opaque' data structures in C too a little more nicely. I think ctypes was started before __sizeof__ became available... so it seems many of it's methods are not updated yet. __sizeof__ is not mandatory, so many extensions have not been updated to support it yet. cheers, On Thu, Dec 23, 2010 at 11:14 AM, Armin Rigo <arigo(a)tunes.org> wrote: > Hi Dima, > > On Wed, Dec 22, 2010 at 11:21 PM, Dima Tisnek <dimaqq(a)gmail.com> wrote: >> --- Comment #4 from Boris Zbarsky (:bz) <bzbarsky(a)mit.edu> 2010-12-22 >> 13:43:23 PST --- >> So what I see this page do, in horizontal mode, is create 17 canvases each of >> which is width="816" height="3587". That means that each of them has a backing >> store of 816*3587*4 = 11,707,968 bytes. So that's about 200MB of memory usage >> right there. >> >> I have no idea why they feel a need for 17 huge canvases, but if they want >> them, that's how much memory they'll take... > > That looks very similar to an issue with PyPy's own GC, in which > ctypes.create_string_buffer() returns objects which tend to be GC'ed > late. That's because the string buffer object in ctypes appears (to > PyPy's GC) to be just a small object, even though it actually > references a potentially large piece of raw memory. Similarly, my > vague guess about the above is that the 17*11MB of memory are hold by > 17 small objects which firefox's GC think don't need to be collected > particularly aggressively. > > > A bientôt, > > Armin. > _______________________________________________ > pypy-dev(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev >

5 6

funding/popularity?
by Dima Tisnek 02 Jan '11

02 Jan '11

On 20 December 2010 19:21, William ML Leslie <william.leslie.ttg(a)gmail.com> wrote: > On 21 December 2010 11:59, Dima Tisnek <dimaqq(a)gmail.com> wrote: >> More visibility for performance achievements would do >> good too. > > Where are pypy's performance achievements *not* visible, but should be? for example http://shootout.alioth.debian.org/ doesn't say which pypy version is used, what options, doesn't have performance figures for multithreaded/multicore also benchmarks are kinda small, most of them are not docuemented, and I couldn't find any info if the same python code was used for cpython and pypy (both shootout and speed pypy) or interpreter-specific verions were used, that is each tweaked for best performance given the known tradeoffs for each implementation.further the most standard benchmarks, pystone, etc. completely ignore the fact that real world programs are large and only a few small paths are execured often. another temp problem with speed pypy is that it's terrubly slow in ff beta. it also occasionally grinds in stable ff and opera, but I guess this can be forgiven for the sake of simplicity / developer effort. if you google for 'python performance' you don't get a single link to pypy on the first page, as a matter of fact, codespeak is poorly indexed, it took me quite some time to get some of my questions answered with a search. also if you look up 'pypy gc' you get a page on codespeak, but to interpret what the data actually means is so far beyond me. a good overview is found in the mainling list http://codespeak.net/pipermail/pypy-dev/2010q1/005757.html then again slowspitfire and spambayes bits are outdated by now. the definitive good thing about pypy is that it's much easier to find out about its inner workings than that of cpython! hopefully a bit more of end-user stuff get known. let's call it pypy public outreach (feature request) > >> Sidetracking... one day when pypy jit/gc/etc are all worked out, how >> hard would it be to use same techniques and most of backends for some >> unrelated language that doesn't have jit yet, e.g. php? > > You know that pypy already has implementations of other languages, > right - Scheme, Javascript, Prolog, IO and smallTalk? They aren't > integrated with the translated pypy-c, but they show that it is not > too difficult to write a runtime for any dynamic language you choose. Oh I didn't know there were so many, and I mistakenly though that js was a target, not implmented langauge. In any case I read somewhere that js support was retired... > >> And how hard >> would it be to marry two dynamic languages, so that modules from one >> could be used in the other? Or that modules written in rpython could >> be used in several langs? > > It's in the "interesting problems" bucket, and the effort required > depends on the level of integration between languages you want. There > are several projects already attempting to do decent integration > between several languages, besides the approach used on the JVM, there > are also GNU Guile, Racket, and Parrot, among others. It might be > worth waiting to see how these different projects pan out, before > spending a bunch of effort just to be an also-ran in the > multi-language runtime market. > > However, implementing more languages in rpython has the side-effect of > propagating the l * o * p problem: it introduces more and more > implementations that then have to be maintained, so good > cross-language integration probably belongs /outside/ pypy itself, so > existing runtimes can hook into it. Makes perfect sense, after all any given other language hardly has the same data model as python. > > But it would be an interesting experiment (to marry the various > interpreters pypy ships with), if you wanted to try it. > > My two cents. > > -- > William Leslie >

5 13

1.4.1 threading
by Dima Tisnek 01 Jan '11

01 Jan '11

Hi, Somehow I'm getting exactly same performance in threaded and non-threaded code (I would expect at least some speedup), Is threading not there in this pypy release in practice or am I missing some argument to pypy? the test case is cpu-bound in python, and possibly gc-bound too

8 17

Fwd: Mercurial support for speed.pypy.org
by Amaury Forgeot d'Arc 27 Dec '10

27 Dec '10

Hi, 2010/12/25 Miquel Torres <tobami(a)googlemail.com> I finally added Mercurial support to Codespeed, following Antonio's > suggestion of parsing the (templated) command line output.Thanks to > everyone that helped! > > I waited for today to give the awesome PyPy team a present. > Thanks! It seems that all changes are listed, even those who happen to be in another branch. Is it possible to filter the list and only display the "default" branch? Cheers, -- Amaury Forgeot d'Arc

2 1

Re: [pypy-dev] dlopen NotImplementedError
by Gary Robinson 26 Dec '10

26 Dec '10

Hmm, I'm thinking the dlopen error might be my problem. I think I built python at /root/pypy, and then moved it to a more proper location. But the error message is referring to /root/pypy, so I'm guessing it has that location hardcoded because that's where I built it? I'm going to rebuild but I wanted to post this message so that you guys don't worry about it (unless rebuilding doesn't help). -- Gary Robinson CTO Emergent Discovery, LLC personal email: garyrob(a)me.com work email: grobinson(a)emergentdiscovery.com Company: http://www.emergentdiscovery.com Blog: http://www.garyrobinson.net

5 5

Mercurial support for speed.pypy.org
by Miquel Torres 25 Dec '10

25 Dec '10

Hi all, I finally added Mercurial support to Codespeed, following Antonio's suggestion of parsing the (templated) command line output.Thanks to everyone that helped! I waited for today to give the awesome PyPy team a present. Merry Christmas! Miquel

2 1

Building PyPy on Windows
by Seung Soo, Ha 23 Dec '10

23 Dec '10

An interesting article came up on python planet: http://www.redmountainsw.com/wordpress/archives/building-pypy-on-windows Maybe with the consent of the original author and some cleaning up, we could add this to official documentation? A simple link on the getting started/download pages won't hurt windows users either.

2 1

Fwd: pypy-dev Re: funding/popularity?
by Dima Tisnek 23 Dec '10

23 Dec '10

---------- Forwarded message ---------- From: Isaac Gouy <igouy2(a)yahoo.com> Dima Tisnek <dimaqq <at> gmail.com> writes: > > On 20 December 2010 19:21, William ML Leslie > <william.leslie.ttg <at> gmail.com> wrote: > > On 21 December 2010 11:59, Dima Tisnek <dimaqq <at> gmail.com> wrote: > >> More visibility for performance achievements would do > >> good too. > > > > Where are pypy's performance achievements *not* visible, but should be? > > for example http://shootout.alioth.debian.org/ > doesn't say which pypy version is used Oh yes it does - http://shootout.alioth.debian.org/u32/compare.php?lang=pypy#about > what options, doesn't have > performance figures for multithreaded/multicore All the Python multiprocessing programs "Failed" with PyPy - http://shootout.alioth.debian.org/u32/measurements.php?lang=pypy > also benchmarks are kinda small, most of them are not docuemented, and > I couldn't find any info if the same python code was used for cpython > and pypy (both shootout and speed pypy) or interpreter-specific > verions were used, that is each tweaked for best performance given the > known tradeoffs for each implementation. The full source code is shown for all the programs - http://shootout.alioth.debian.org/u32/program.php?test=spectralnorm〈=pypy&i… http://shootout.alioth.debian.org/u32/program.php?test=spectralnorm〈=python… The GZip byte size is shown for each program, in this case 341 for both programs. Maybe the benchmarks game shows more information than you realize.

1 0

dlopen NotImplementedError
by Gary Robinson 22 Dec '10

22 Dec '10

So, as of today, I'm starting to do as much of my day-to-day work in pypy as I can. The missing numpy is a problem, but one I can work around a enough amount of the time to make this effort worth my while. I don't know how, or even whether, I should report certain kinds of problems I might run into. I would appreciate some feedback on that. I want to help by giving feedback in whatever form, and to what degree, the pypy devs desire. For example, I can run some scripts in pypy 1.4 but not in pypy fast-forward because of an error involving dlopen(). Other scripts are unaffected by this problem, depending on what needs to be imported. When I try to run fast-forward in interactive mode, I get the same error, which is pasted below. Python 2.7.0 (8c2427fa1a01, Dec 19 2010, 15:58:12) [PyPy 1.4.0 with GCC 4.1.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "app_main.py", line 522, in run_it File "/etc/pythonstart", line 7, in <module> import readline File "/root/pypy/lib_pypy/readline.py", line 9, in <module> File "/root/pypy/lib_pypy/pyrepl/readline.py", line 33, in <module> File "/root/pypy/lib_pypy/pyrepl/unix_console.py", line 25, in <module> File "/root/pypy/lib_pypy/pyrepl/curses.py", line 30, in <module> File "/root/pypy/lib_pypy/_minimal_curses.py", line 12, in <module> File "/root/pypy/lib-python/modified-2.7.0/ctypes/__init__.py", line 441, in <module> File "/root/pypy/lib-python/modified-2.7.0/ctypes/__init__.py", line 353, in __init__ File "/root/pypy/lib_pypy/_ctypes/dll.py", line 7, in dlopen NotImplementedError: dlopen(None) And now for something completely different: ``- do you know berlin? - all of it? - well, just berlin'' Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "/root/pypy/lib_pypy/_pypy_interact.py", line 27, in interactive_console File "/root/pypy/lib_pypy/pyrepl/simple_interact.py", line 27, in <module> File "/root/pypy/lib_pypy/pyrepl/readline.py", line 33, in <module> File "/root/pypy/lib_pypy/pyrepl/unix_console.py", line 25, in <module> File "/root/pypy/lib_pypy/pyrepl/curses.py", line 30, in <module> File "/root/pypy/lib_pypy/_minimal_curses.py", line 12, in <module> File "/root/pypy/lib-python/modified-2.7.0/ctypes/__init__.py", line 441, in <module> File "/root/pypy/lib-python/modified-2.7.0/ctypes/__init__.py", line 353, in __init__ File "/root/pypy/lib_pypy/_ctypes/dll.py", line 7, in dlopen NotImplementedError: dlopen(None) -- Gary Robinson CTO Emergent Discovery, LLC personal email: garyrob(a)me.com work email: grobinson(a)emergentdiscovery.com Company: http://www.emergentdiscovery.com Blog: http://www.garyrobinson.net

1 0