Mailman 3 September 2011 - pypy-dev

Misc news
by Armin Rigo Sept. 5, 2011

Sept. 5, 2011

Hi all, Misc news: * the windows buildbot should now upload its nightly results, similarly to the other buildbots. You can find it together with builds for other platforms at http://buildbot.pypy.org/nightly/trunk/ . There seem to be still Windows-specific bugs, though. * stackless support could do with some volunteer work now --- in particular, lib_pypy/stackless.py could be refactored to use directly continulets, in a similar way to lib_pypy/greenlet.py. I am ready to explain a bit more … [View More]

3 5

Re: [pypy-dev] PyCON Finland has just announced its opening
by Laura Creighton Sept. 4, 2011

Sept. 4, 2011

In a message of Sun, 04 Sep 2011 19:40:01 +0200, Maciej Fijalkowski writes: >It's a surprisingly good time for me. I stay in EU until 14th of Nov. Cool! Been too long since I saw you. Laura

1 0

Re: [pypy-dev] PyCON Finland has just announced its opening
by Laura Creighton Sept. 4, 2011

Sept. 4, 2011

In a message of Sat, 03 Sep 2011 13:53:26 +0200, Armin Rigo writes: >Hi Laura, > >On Fri, Sep 2, 2011 at 4:31 PM, Laura Creighton <lac(a)openend.se> wrote: >> It's not that far away for some of us, Finland is beautiful, and they >> still have space for talks and sprints. > >I appreciate the "bait" factor in this message :-) But as >importantly, I think that you should tell this list about planning to >organize a sprint in GÃ¶teborg around this time, so … [View More]

2 1

your thoughts on low level optimizations
by Gertjan van Zwieten Sept. 3, 2011

Sept. 3, 2011

Dear all, Following today's blog post [1] about wrapping C++ libraries, I would like to take the opportunity to get some opinions from the PyPy community on a related topic. I hope this mailing list is the appropriate place for this sort of discussion. My name is Gertjan van Zwieten and I have been a Python enthusiast since long. I have used it extensively in the context of my PhD project, which was related to Finite Element modelling, and I have recently taken steps together with a friend of … [View More]mine towards continuing on this path commercially [2]. We intend to design our own Finite Element framework, which we aim to be open source, written largely in Python. It is the 'largely' that I would like to discuss here. I have always argued to people that, yes, it is possible to do computationally intensive things in an interpreted language, as long as the majority of work is done by compiled components. For me the obvious example of that is Numpy, which provides a rather specific data type and a set of vectorized operations, as well as an interface with optimized linear algebra libraries. This leaves Python as a convenient and very powerful glue language to connect these optimized components into something purposeful. With that conceptualization in mind, my preferred route for optimizing code was to identify critical components of generic nature, and re-implement them as a plain C module using the Python API. So this is how I used to look at things. I would say that things were nice and clear this way, until PyPy started throwing stones in the water. I have not so far used PyPy in any actual application, mainly (if not only) because of lacking Numpy support. But I can see it holds great promise, especially for the computational community (I am also eagerly following the transactional memory developments) and I anticipate making the switch after it matures a little bit further. However, that does put me in the situation that, currently, I am not sure anymore what things to implement in C, if any, and in what form. Right now I am prototyping our framework using only Numpy and pure Python. I avoid premature optimization as much as possible, and in fact at times find myself sacrificing speed for elegance, arguing that I will be able to bring back efficiency later on in a deeper layer. For example, rather than adopting a numbering scheme for the many computational elements such as is common in Finite Element implementations, I have moved to a more object oriented approach that is more flexible and allows for more error checking, but that forces me to put in dictionaries what used to be Numpy arrays. Obviously dictionaries were not meant to be numeric data structures, and it's a typical component that I would try to implement efficiently in C eventually. With PyPy at the horizon, I am not so sure anymore. For one I'm not sure if PyPy will ever be able to use the C module, or use it efficiently; I could understand if support exists merely for sake of compatibility. I am also not certain if I should maybe forget about C altogether and rely on the JIT to compile the Python for loops that I have always tried to avoid. Would you go as far as to say that there will be no more reason for low level programming whatsoever? Or would you advise to write the component in RPython and use the translator to compile it? With my poor overview of these things, there are very few arguments that I can formulate in favour or against any of these options. Regarding today's blog post, I have the feeling that this is meant more for wrapping existing C++ libraries than for starting new ones, is that correct? Or if not, and it is, in fact, an option to consider, will this be able to work in CPython, too? That would make the transition a bit easier, obviously. I am very interested what your views are on this topic of optimizations. Best regards and thanks a lot for working on PyPy! Gertjan [1] http://morepypy.blogspot.com/2011/08/wrapping-c-libraries-with-reflection.h… [2] http://hvzengineering.nl [View Less]

5 9

C-based numpy's nonzero() has the same performance as its vanilla Python implementation running pypy
by Valery Khamenya Sept. 3, 2011

Sept. 3, 2011

isn't that cool? my 1-dimensional vanilla Python implementation was: lambda v: [i for i,e in enumerate(v) if e != 0] best regards -- Valery A.Khamenya

1 0

PyCON Finland has just announced its opening
by Laura Creighton Sept. 3, 2011

Sept. 3, 2011

http://fi.pycon.org/2011/#schedule It's not that far away for some of us, Finland is beautiful, and they still have space for talks and sprints. Laura

2 1

Solaris Support?
by Peter Kruse Sept. 2, 2011

Sept. 2, 2011

Hello, I'd like to compile PyPy under Solaris/Sparc. But it looks that this is not supported. Right now when I run "python2.7 translate.py -O2" as suggested on http://pypy.org/download.html I get an exception: [version:WARNING] Errors getting Mercurial information: Not running from a Mercurial repository! [platform:msg] Setting platform to 'host' cc=None [translation:info] Translating target as defined by targetpypystandalone Traceback (most recent call last): File "translate.py", line … [View More]

2 3

Windows build possibility on Amazon
by Andrew Dalke Sept. 2, 2011

Sept. 2, 2011

I was talking with Laura and she said there's still no good way to get pypy builds for Windows. I mentioned that Amazon EC2 has Windows available for rent http://aws.amazon.com/windows/ There's prebuilt disk images for Windows with a Django install http://aws.amazon.com/amis/Microsoft-Windows/7235836237155671 so Python would come with the image. You all probably want Large Instance: 7.5 GB of memory (small is 1.7, extra large is 15) It's available for $0.48 per hour . Andrew dalke(a)dalkescientific.com

5 5

Here's a fun one...
by Dino Viehland Sept. 1, 2011

Sept. 1, 2011

This came up on an internal discussion, I thought it was fun, especially given that we all behave differently: Paste this into the REPL: class PS1(object): def __init__(self): self.count = 0 def __str__(self): self.count += 1 return "%d >>>" % self.count import sys sys.ps1 = PS1() CPython - calls __str__ Jython - calls __repr__ IronPython - ignores ps1 PyPy - unsupported operand type for unary buffer: 'PS1' (note I don't necessarily have the latest versions for everyone)

3 6

Failing App-Level-Test test_posix2.py
by Juergen Boemmels Sept. 1, 2011

Sept. 1, 2011

Hi, Since some weeks in one Applevel-test is failing consitently on the buildbot: module/posix/test/test_posix2.py with the failure: if sys.platform.startswith('linux'): > assert hasattr(st, 'st_rdev') E assert hasattr(posix.stat_result(st_mode=33152,st_ino=1588275L, st_dev=64256L, st_nlink=1,s...integer_atime=1314759734, … [View More]_integer_mtime=1314759734,_integer_ctime=1314759734), 'st_rdev') module/posix/test/test_posix2.py:135: AssertionError ================ 1 failed, 77 passed, 6 skipped in 4.80 seconds ================ So it seems that st_rdev is not available I tracked this problem down to a chicken-egg problem. The relevant portion is here in pypy/rpython/module/ll_os_stat.py: # for now, check the host Python to know which st_xxx fields exist STAT_FIELDS = [(_name, _TYPE) for (_name, _TYPE) in ALL_STAT_FIELDS if hasattr(os.stat_result, _name)] As pypy is meanwhile build with pypy, relying on the host python is not a good idea. A pypy without st_rdev will again build only a new pypy without st_rdev, even if the platform supports st_rdev. As an experiment I bootstrapped pypy again with python an the error disappered # python translate.py -O2 targetpypystandalone.py # ./pypy-c ../../../pytest.py ../../module/posix/test/test_posix2.py -A ========================= test session starts ========================== platform linux2 -- Python 2.7.1[pypy-1.6.0] -- pytest-2.1.0.dev4 pytest-2.1.0.dev4 from /home/boemmels/src/pypy/pytest.pyc collected 80 items ../../module/posix/test/test_posix2.py ...........................................................s............ ........ ================= 79 passed, 1 skipped in 5.07 seconds ================= This bootstrapped pypy-c is also cappable of rebuilding an pypy with st_rdev enabled. But this is not a clean solution. I think the way to go is to no longer rely on the Host-Python, but use the appropiate configure magic. Something like #ifdef HAVE_STRUCT_STAT_ST_RDEV but in a pythonic way. Unfortunatly I'm not familar with pypy's configuration system so I got stuck here. Can anybody tell me how to test for available struct members like AC_CHECK_MEMBERS([struct stat.st_rdev]) in autoconf? Regards juergen ___________________________________________________________ Schon gehört? WEB.DE hat einen genialen Phishing-Filter in die Toolbar eingebaut! http://produkte.web.de/go/toolbar [View Less]

2 1