Mailman 3 June 2012 - pypy-dev

change of strategy for the py3k branch?
by Antonio Cuni June 5, 2012

June 5, 2012

Hi all, after some months of work on the py3k branch, I realized that the current strategy/workflow does not scale well and thus I'd like to change it. For those who are not aware, currently we have the default branch where the main development is done, and which includes code for both the rpython translator toolchain and the python 2 interpreter. The py3k branch does not touch the translator toolchain, but modifies the python 2 interpreter to make it a py3k interpreter. These changes are actually destroying the python 2 semantics, which means that the long term goal is to never merge py3k to default, but keeping the development in parallel, and regularly merge default into py3k to make sure that py3k gets benefits of the various improvements in the JIT, GC, etc. In the past months, I ended up spending a considerable amount of time in resolving merge conflicts. This happens all the time that someone modifies something in the python 2 interpreter, for example to apply a new cool optimizations. While on the one hand it is cool to automatically have new cool optimizations on py3k, on the other hand it is blocker which stops me to work on it effectively. After a bit of discussion on IRC, I propose to solve this problem by detaching the development of the py3k interpreter from the development of the python 2 one. Pros: - faster development of py3k - lower entry barrier for new contributors, because the relationship between the various parts will be much simpler - it will be straightforward to apply the new features of the translator toolchain to the py3k branch - it will be easier to split the toolchain from the actual interpreter the day we will finally decide to do it Cons: - we will need to manually port the optimizations done in the interpreter on the default branch to py3k. Note that right now it's now really "automatic" anyway, because merging is painful. If we decide to go for this route, the next question is: where to store the code? I think there are two main solutions: 1) add a new "pypy/py3k" directory where to copy all the relevant modules. E.g. pypy/py3k/interpreter, pypy/py3k/objspace/std, "pypy/py3k/modules. 2) start a completely new repository which contains only the code for py3k. Solution (2) is better and cleaner in theory. However I fear it would soon become a mess to handle, because every change in the translator toolchain would potentially break py3k. I don't want a situation in which we say "yes, you can build py3k but only if you take revision XXX and you use revision YYY of the toolchain, unless the phase of the moon is empty". Solution (1) is more practical and it would probably lead to less problems in the short term. For now, I would still keep the code in the py3k branch, so the normal development of pypy would not be affected. Before doing it, I'd like to hear opinions and comments, in particular of people who already worked on py3k and/or are generally interested in it. Please be constructive :-). ciao, Anto

10 30

New pypy website update
by ante June 5, 2012

June 5, 2012

Hi all, Here are some of the major updates done to the new website in recent months. - Download links automatically updated from bitbucket/pypy download page. - Supported libs automatically updated from bitbucket/pypy wiki. - Fixed usability/linking issues with the sub section navigation. - Code highlighting - Write content in either ReST/raw text/HTML. - Working twitter feed. - Better performance overall. - Migrated from ep.io to dotCloud. www: http://pypy-nikuda.dotcloud.com/ hg: https://bitbucket.org/pypy/pypy-site wiki: https://bitbucket.org/nikuda/pypy-site/wiki/Home To address some of the issues brought up last time the new website was discussed: One of the reasons for updating the design of the website was to make it more accessible to end-users, while having the deeper technical details available for those that need it. This is the reason behind implementing the 'hidden' until wanted, or dig down approach in the structure of the website. I understand this can seem unnecessary to people who are very used to the project and how it works. Even though I don't see a problem with it, it's only one possible solution and we can change things around if we agree that this particular implementation is not suited. The website is not finished and there are a few things that I want to change with it, but I do believe there's a good working base there that can be expanded on and improved. Approach is definitively an iterative one, so feel free add suggestions here or in the wiki. Thanks, Ante

3 3

reusing test classess from another module's test directory?
by wlavrijsen＠lbl.gov June 4, 2012

June 4, 2012

Hello, in the PyObject* converter and executor, I use cpyext to get hold of a PyObject* to send to/receive from the C++ world. That works okay, but it proves rather hard to test. I now have a test that does what I want, based on AppTestCpythonExtensionBase, where pretty much all the heavy lifting is done, saving me tons of work. What is the recommended way to grab test bases from the test area of another module? Right now, I'm doing the moderately ugly: sys.path.append(str(py.path.local(autopath.pypydir).join('module', 'cpyext', 'test'))) from test_cpyext import AppTestCpythonExtensionBase which works fine, but surely there must be a better way? Thanks, Wim -- WLavrijsen(a)lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net

2 2

IOError with getpass() in django
by Gelonida N June 4, 2012

June 4, 2012

Hi, I reposted as I noticed, that my inital subject line doesn't really indicate what's the issue. ------------------------------------------------------------------- I wanted to make my very first test with pypy by using django and sqlite3 on a Linux host (Ubuntu 12.04) when running ./manage.py syncdb I'm asked to enter a password and pypy aborts as soon as I press enter. > File "/home/klausf/mypypy/site-packages/django/contrib/auth/management/commands/createsuperuser.py", line 109, in handle > password = getpass.getpass() > File "/usr/lib/pypy/lib-python/2.7/getpass.py", line 74, in unix_getpass > stream.flush() # issue7208 > IOError: [Errno 29] Illegal seek: '<fdopen>' Version info: Python 2.7.2 (1.8+dfsg-2, Feb 19 2012, 19:18:08) [PyPy 1.8.0 with GCC 4.6.2] Django version: 1.3.1 I even found a bug, which seems related ( https://bugs.pypy.org/issue872 ) However I don't understand the impact and what this means exactly. Should I try to install another pypy (and if yes, how can I install it next to an existing one on Ubuntu 12.04)? Thanks in advance for your help.

3 3

[Repost] PyPy 1.8 50% slower than PyPy 1.7 on my test case
by Laurent Vaucher June 4, 2012

June 4, 2012

Sorry to repost, but does anyone have an idea about what I could do to track down the source of the slowdown? Should I run with some trace to try to compare how different parts behave? How could I do that? Thanks, Laurent. Hi and first of all, thanks for that great project. Now to my "problem". I'm doing some puzzle-solving, constraint processing with Python and on my particular program, PyPy 1.8 showed a 50% increase in running time over PyPy 1.7. I'm not doing anything fancy (no numpy, itertools, etc.). The program is here: https://github.com/slowfrog/hexiom To reproduce: - fetch hexiom2.py and level36.txt from github - run 'pypy hexiom2.py -sfirst level36.txt' On my machine (Windows 7) the timings are the following: Python 2.6.5/Windows 32 3m35s Python 2.7.1 (930f0bc4125a, Nov 27 2011, 11:58:57) [PyPy 1.7.0 with MSC v.1500 32 bit] 31s Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 18:31:47) [PyPy 1.8.0 with MSC v.1500 32 bit] 48s I'm using the default options. Do you have any idea what could be causing that? Thanks, Laurent.

3 5

Re: [pypy-dev] pypy-dev Digest, Vol 13, Issue 21
by exarkun＠twistedmatrix.com June 3, 2012

June 3, 2012

On 03:11 pm, pypy-dev-request(a)python.org wrote: > >Hi all, > >Fijal and me would like to raise interest among various groups of >people about building a better ctypes replacement for Python. > >The general background first, at least as far as we know it. People >generally agree that CPython extension modules are not extremely hard >to write, but also not the ultimate solution in the matter of >interfacing with C code. Obviously you need to know and write C code, >and learn the large CPython C API. Moreover a CPython extension >module is rather specific to CPython or even a particular version of >it. Even if you are lucky and your CPython extensions work on all 2.x >versions, it won't work unmodified on Python 3.x, and be slow (or not >work at all) on PyPy. (I'm leaving Jython and IronPython out of the >picture here but it's very far from ideal for them too.) > >The working alternatives nowadays are Cython and ctypes. For Cython >you have to learn a whole new language; we can argue infinitely about >preferences, but from my point of view it's mostly just a nicer way to >write CPython C extensions using --- instead of C --- a custom >language. About ctypes, I would say that the fact that its use >remains marginal is enough to show that its mission kind of failed. >Its problems are more than one, but most importantly the level at >which you describe the information is slightly too low - it's >ABI-level instead of API-level. It does not make a difference for some >libraries, but for some it does make a huge difference, like when you >use macros. We would like something that can be potentially used to >wrap almost all C libraries. > >So we would like to propose something (a bit more publically than the >two subprojects of PyPy attempting to do that --- did you know about >their existence?). The goal would be to come up with a >better-designed project. "Better designed" in my vocabulary implies >"simpler". I'm starting in this mailing list but the idea is to try >to move it in its own project, and to develop at the same time at >least a CPython 2.7 and 3.x and PyPy implementation. > >The simplest FFI we know of for a high-level language is LuaJIT's FFI. >If you are interested, read the four pages starting at >http://luajit.org/ext_ffi.html . A crucial design decision was to >avoid making use of tons of objects representing the C types; instead, >the user mostly specifies C types as plain strings, simplifying the >basic API a lot. Hi Armin, I don't think it's true that using strings instead of types (or other rich objects) simplifies anything. Quite the opposite, it takes all of the complexity which must exist and throws a huge wall up to prevent anyone from understanding it without undertaking a huge amount of extra work. C is C, whether you hide the representation of its structure in a string or whether you expose that representation as objects with useful, easy to use APIs. Jean-Paul

10 21

Re: [pypy-dev] [pypy-commit] pypy win_ffi: fix test for jit
by Antonio Cuni June 2, 2012

June 2, 2012

Hello Matti, On 06/01/2012 03:25 PM, mattip wrote: > @@ -551,6 +552,9 @@ > except ValueError, e: > assert e.message == 'Procedure called with not enough ' + \ > 'arguments (8 bytes missing) or wrong calling convention' > + except LLException: > + #jitted code raises this > + pass > else: > assert 0, 'wrong calling convention should have raised' I think you should at least check that the LLException you get is of the right type, to avoid hiding an unrelated one you might get by chance. ciao, Anto

1 0

PyPy build bots?
by Timothy Baldridge June 2, 2012

June 2, 2012

I'm at the point where I'd like to start somewhat regular builds of my pypy branch complete with JIT optimization. I tried downloading the build bot but ran into some errors. Does pypy have a build farm that branches can use? Any suggestions on the path-of-least resistance to fire off nightly builds? Thanks, Timothy

2 1

PyPy PPA
by Alex Gaynor June 1, 2012

June 1, 2012

Hey all, I'm hoping the people behind https://launchpad.net/~pypy/+archive/ppa are hear :) I'm wondering what the status of this (or any other PPAs) are, and whether it'd be possible to get a PPA that works with Ubuntu 12.04. The people behind http://travis-ci.org/ which is an awesome CI service, would like to be able to deploy PyPy via a deb on 12.04. Thanks, Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

3 2