Mailman 3 September 2011 - pypy-dev

Misc news
by Armin Rigo 05 Sep '11

05 Sep '11

Hi all, Misc news: * the windows buildbot should now upload its nightly results, similarly to the other buildbots. You can find it together with builds for other platforms at http://buildbot.pypy.org/nightly/trunk/ . There seem to be still Windows-specific bugs, though. * stackless support could do with some volunteer work now --- in particular, lib_pypy/stackless.py could be refactored to use directly continulets, in a similar way to lib_pypy/greenlet.py. I am ready to explain a bit more on irc how continulets work, but I don't have enough motivation to do the complete refactoring myself. It's also a nice entry-level task for someone who wants to. A bientôt, Armin.

3 5

Re: [pypy-dev] PyCON Finland has just announced its opening
by Laura Creighton 04 Sep '11

04 Sep '11

In a message of Sun, 04 Sep 2011 19:40:01 +0200, Maciej Fijalkowski writes: >It's a surprisingly good time for me. I stay in EU until 14th of Nov. Cool! Been too long since I saw you. Laura

1 0

Re: [pypy-dev] PyCON Finland has just announced its opening
by Laura Creighton 04 Sep '11

04 Sep '11

In a message of Sat, 03 Sep 2011 13:53:26 +0200, Armin Rigo writes: >Hi Laura, > >On Fri, Sep 2, 2011 at 4:31 PM, Laura Creighton <lac(a)openend.se> wrote: >> It's not that far away for some of us, Finland is beautiful, and they >> still have space for talks and sprints. > >I appreciate the "bait" factor in this message :-) But as >importantly, I think that you should tell this list about planning to >organize a sprint in GÃ¶teborg around this time, so that we can check >with interested people if the dates seem to work. >A bientÃ´t, > >Armin. Ok, let's have a sprint in GÃ¶teborg around this time. Also of interest is FSCons: http://fscons.org/ Nov 11-13 who already have been promised a talk by somebody from PyPy. So a reasonable thing to do is to Finland for Oct 17-18, and have a sprint sometime in between. A tricky bit is that we need to be available for an interview with Vinnova in Stockholm on the 24th or the 28th of October. I don't know how much notice they will give us -- so my tentative plan is to start a sprint in GÃ¶teborg in the first week of November. Is this good for other people? Would others like a different start or end time, or ... Laura

2 1

your thoughts on low level optimizations
by Gertjan van Zwieten 03 Sep '11

03 Sep '11

Dear all, Following today's blog post [1] about wrapping C++ libraries, I would like to take the opportunity to get some opinions from the PyPy community on a related topic. I hope this mailing list is the appropriate place for this sort of discussion. My name is Gertjan van Zwieten and I have been a Python enthusiast since long. I have used it extensively in the context of my PhD project, which was related to Finite Element modelling, and I have recently taken steps together with a friend of mine towards continuing on this path commercially [2]. We intend to design our own Finite Element framework, which we aim to be open source, written largely in Python. It is the 'largely' that I would like to discuss here. I have always argued to people that, yes, it is possible to do computationally intensive things in an interpreted language, as long as the majority of work is done by compiled components. For me the obvious example of that is Numpy, which provides a rather specific data type and a set of vectorized operations, as well as an interface with optimized linear algebra libraries. This leaves Python as a convenient and very powerful glue language to connect these optimized components into something purposeful. With that conceptualization in mind, my preferred route for optimizing code was to identify critical components of generic nature, and re-implement them as a plain C module using the Python API. So this is how I used to look at things. I would say that things were nice and clear this way, until PyPy started throwing stones in the water. I have not so far used PyPy in any actual application, mainly (if not only) because of lacking Numpy support. But I can see it holds great promise, especially for the computational community (I am also eagerly following the transactional memory developments) and I anticipate making the switch after it matures a little bit further. However, that does put me in the situation that, currently, I am not sure anymore what things to implement in C, if any, and in what form. Right now I am prototyping our framework using only Numpy and pure Python. I avoid premature optimization as much as possible, and in fact at times find myself sacrificing speed for elegance, arguing that I will be able to bring back efficiency later on in a deeper layer. For example, rather than adopting a numbering scheme for the many computational elements such as is common in Finite Element implementations, I have moved to a more object oriented approach that is more flexible and allows for more error checking, but that forces me to put in dictionaries what used to be Numpy arrays. Obviously dictionaries were not meant to be numeric data structures, and it's a typical component that I would try to implement efficiently in C eventually. With PyPy at the horizon, I am not so sure anymore. For one I'm not sure if PyPy will ever be able to use the C module, or use it efficiently; I could understand if support exists merely for sake of compatibility. I am also not certain if I should maybe forget about C altogether and rely on the JIT to compile the Python for loops that I have always tried to avoid. Would you go as far as to say that there will be no more reason for low level programming whatsoever? Or would you advise to write the component in RPython and use the translator to compile it? With my poor overview of these things, there are very few arguments that I can formulate in favour or against any of these options. Regarding today's blog post, I have the feeling that this is meant more for wrapping existing C++ libraries than for starting new ones, is that correct? Or if not, and it is, in fact, an option to consider, will this be able to work in CPython, too? That would make the transition a bit easier, obviously. I am very interested what your views are on this topic of optimizations. Best regards and thanks a lot for working on PyPy! Gertjan [1] http://morepypy.blogspot.com/2011/08/wrapping-c-libraries-with-reflection.h… [2] http://hvzengineering.nl

5 9

C-based numpy's nonzero() has the same performance as its vanilla Python implementation running pypy
by Valery Khamenya 03 Sep '11

03 Sep '11

isn't that cool? my 1-dimensional vanilla Python implementation was: lambda v: [i for i,e in enumerate(v) if e != 0] best regards -- Valery A.Khamenya

1 0

PyCON Finland has just announced its opening
by Laura Creighton 03 Sep '11

03 Sep '11

http://fi.pycon.org/2011/#schedule It's not that far away for some of us, Finland is beautiful, and they still have space for talks and sprints. Laura

2 1

Solaris Support?
by Peter Kruse 02 Sep '11

02 Sep '11

Hello, I'd like to compile PyPy under Solaris/Sparc. But it looks that this is not supported. Right now when I run "python2.7 translate.py -O2" as suggested on http://pypy.org/download.html I get an exception: [version:WARNING] Errors getting Mercurial information: Not running from a Mercurial repository! [platform:msg] Setting platform to 'host' cc=None [translation:info] Translating target as defined by targetpypystandalone Traceback (most recent call last): File "translate.py", line 324, in <module> ... File "/net/sdevc01/export/data/soft/opensource/source/pypy/pypy-pypy-release-1.6/pypy/module/sys/version.py", line 17, in <module> elif platform.cc.startswith('gcc'): AttributeError: 'NoneType' object has no attribute 'startswith' As I have python2.7, gcc 4.6.1 and libffi it seems that the requirements for PyPy should be fulfilled, so what do I have to do to compile it under Solaris? Thanks, Peter ps: I'm not subscribed to this list, it would be very kind if you cc me on replies.

2 3

Windows build possibility on Amazon
by Andrew Dalke 02 Sep '11

02 Sep '11

I was talking with Laura and she said there's still no good way to get pypy builds for Windows. I mentioned that Amazon EC2 has Windows available for rent http://aws.amazon.com/windows/ There's prebuilt disk images for Windows with a Django install http://aws.amazon.com/amis/Microsoft-Windows/7235836237155671 so Python would come with the image. You all probably want Large Instance: 7.5 GB of memory (small is 1.7, extra large is 15) It's available for $0.48 per hour . Andrew dalke(a)dalkescientific.com

5 5

Here's a fun one...
by Dino Viehland 01 Sep '11

01 Sep '11

This came up on an internal discussion, I thought it was fun, especially given that we all behave differently: Paste this into the REPL: class PS1(object): def __init__(self): self.count = 0 def __str__(self): self.count += 1 return "%d >>>" % self.count import sys sys.ps1 = PS1() CPython - calls __str__ Jython - calls __repr__ IronPython - ignores ps1 PyPy - unsupported operand type for unary buffer: 'PS1' (note I don't necessarily have the latest versions for everyone)

3 6

Failing App-Level-Test test_posix2.py
by Juergen Boemmels 01 Sep '11

01 Sep '11

Hi, Since some weeks in one Applevel-test is failing consitently on the buildbot: module/posix/test/test_posix2.py with the failure: if sys.platform.startswith('linux'): > assert hasattr(st, 'st_rdev') E assert hasattr(posix.stat_result(st_mode=33152,st_ino=1588275L, st_dev=64256L, st_nlink=1,s...integer_atime=1314759734, _integer_mtime=1314759734,_integer_ctime=1314759734), 'st_rdev') module/posix/test/test_posix2.py:135: AssertionError ================ 1 failed, 77 passed, 6 skipped in 4.80 seconds ================ So it seems that st_rdev is not available I tracked this problem down to a chicken-egg problem. The relevant portion is here in pypy/rpython/module/ll_os_stat.py: # for now, check the host Python to know which st_xxx fields exist STAT_FIELDS = [(_name, _TYPE) for (_name, _TYPE) in ALL_STAT_FIELDS if hasattr(os.stat_result, _name)] As pypy is meanwhile build with pypy, relying on the host python is not a good idea. A pypy without st_rdev will again build only a new pypy without st_rdev, even if the platform supports st_rdev. As an experiment I bootstrapped pypy again with python an the error disappered # python translate.py -O2 targetpypystandalone.py # ./pypy-c ../../../pytest.py ../../module/posix/test/test_posix2.py -A ========================= test session starts ========================== platform linux2 -- Python 2.7.1[pypy-1.6.0] -- pytest-2.1.0.dev4 pytest-2.1.0.dev4 from /home/boemmels/src/pypy/pytest.pyc collected 80 items ../../module/posix/test/test_posix2.py ...........................................................s............ ........ ================= 79 passed, 1 skipped in 5.07 seconds ================= This bootstrapped pypy-c is also cappable of rebuilding an pypy with st_rdev enabled. But this is not a clean solution. I think the way to go is to no longer rely on the Host-Python, but use the appropiate configure magic. Something like #ifdef HAVE_STRUCT_STAT_ST_RDEV but in a pythonic way. Unfortunatly I'm not familar with pypy's configuration system so I got stuck here. Can anybody tell me how to test for available struct members like AC_CHECK_MEMBERS([struct stat.st_rdev]) in autoconf? Regards juergen ___________________________________________________________ Schon gehört? WEB.DE hat einen genialen Phishing-Filter in die Toolbar eingebaut! http://produkte.web.de/go/toolbar

2 1