Mailman 3 February 2012 - pypy-dev

PyPy/NumPyPy sprint(s) in Leipzig, Germany?
by Mike Müller Feb. 21, 2012

Feb. 21, 2012

Hi, I am just wondering if anybody is interested in sprinting on PyPy and in particular NumPyPy in Leipzig sometime in 2012. I can offer working space for up to 12 people with Wi-Fi as well as some basic catering (hot and cold drinks, snacks, pizza etc.) for a few days (up to a week). Accommodation in Leipzig is very reasonably priced. For example, there is a decent hotel very close to the venue for 35 Euros/night (single) or 50 Euros/night (double) including breakfast. Being a logistics … [View More]

5 31

Re: [pypy-dev] Poor performance with custom bytecode
by Antonio Cuni Feb. 18, 2012

Feb. 18, 2012

On 02/17/2012 02:27 PM, Timothy Baldridge wrote: >> In pypy we have a custom opcode to call methods, which is much faster than >> LOAD_ATTR/CALL_FUNCTION. See e.g. how this piece of code gets compiled: >> > > Excellent! I was unaware of this. Just last night I started > abstracting some bytecode generation to support both 2.6 and 2.7, so > it won't be hard to slot in improvements for pypy. > > From there, I'll start looking into jitviewer. I suggest to look … [View More]

6 14

Poor performance with custom bytecode
by Timothy Baldridge Feb. 17, 2012

Feb. 17, 2012

Last night, I was finally able to get enough of core.clj implemented to run a basic factorial program in clojure-py (https://github.com/halgari/clojure-py). In this project we use byteplay to generate bytecode for clojure routines, and run the entire language off the python vm. In general, I've been very impressed with the performance of pypy, but this factorial program takes about 2x longer to complete than the same routine running on CPython. Perhaps someone can give me some pointers? The … [View More]core of the whole test is the factorial function: (ns clojure.examples.factorial) (defn fact [x] (loop [n x f 1] (if (= n 1) f (recur (dec n) (* f n))))) (defn test [times] (loop [rem times] (if (> rem 0) (do (fact 20000) (print rem) (recur (dec rem)))))) (test 20) It all seems to match normal python bytecode, with one exception: the implementation of *, = and >. In clojure these functions can take 0 to n arguments and perform different logic based on the results. Our solution? to stuff the arguments into __argsv__ and then perform a len() on this argument, and jump to different code blocks based on the length of the input tuple. Now I know this code may not be super fast, but it seems to work fine in CPython. Could this be the pain point for us in pypy? Any thoughts/ideas? clojure.examples.factorial=> (dis.dis *) 0 0 LOAD_FAST 0 (__argsv__) 3 LOAD_ATTR 0 (__len__) 6 CALL_FUNCTION 0 9 LOAD_CONST 1 (0) 12 COMPARE_OP 2 (==) 15 POP_JUMP_IF_FALSE 22 18 LOAD_CONST 2 (1) 21 RETURN_VALUE >> 22 LOAD_FAST 0 (__argsv__) 25 LOAD_ATTR 0 (__len__) 28 CALL_FUNCTION 0 31 LOAD_CONST 2 (1) 34 COMPARE_OP 2 (==) 37 POP_JUMP_IF_FALSE 54 40 LOAD_FAST 0 (__argsv__) 43 LOAD_CONST 1 (0) 46 BINARY_SUBSCR 47 STORE_FAST 1 (x) 50 LOAD_FAST 1 (x) 53 RETURN_VALUE >> 54 LOAD_FAST 0 (__argsv__) 57 LOAD_ATTR 0 (__len__) 60 CALL_FUNCTION 0 63 LOAD_CONST 3 (2) 66 COMPARE_OP 2 (==) 69 POP_JUMP_IF_FALSE 100 72 LOAD_FAST 0 (__argsv__) 75 LOAD_CONST 1 (0) 78 BINARY_SUBSCR 79 STORE_FAST 1 (x) 82 LOAD_FAST 0 (__argsv__) 85 LOAD_CONST 2 (1) 88 BINARY_SUBSCR 89 STORE_FAST 2 (y) 1010 92 LOAD_FAST 1 (x) 95 LOAD_FAST 2 (y) 98 BINARY_MULTIPLY 99 RETURN_VALUE >> 100 LOAD_FAST 0 (__argsv__) 103 LOAD_ATTR 0 (__len__) 106 CALL_FUNCTION 0 109 LOAD_CONST 3 (2) 112 COMPARE_OP 5 (>=) 115 POP_JUMP_IF_FALSE 173 118 LOAD_FAST 0 (__argsv__) 121 LOAD_CONST 1 (0) 124 BINARY_SUBSCR 125 STORE_FAST 1 (x) 128 LOAD_FAST 0 (__argsv__) 131 LOAD_CONST 2 (1) 134 BINARY_SUBSCR 135 STORE_FAST 2 (y) 138 LOAD_FAST 0 (__argsv__) 141 LOAD_CONST 3 (2) 144 SLICE+1 145 STORE_FAST 3 (more) 1012 148 LOAD_GLOBAL 1 (reduce1) 151 LOAD_GLOBAL 2 (*) 154 LOAD_GLOBAL 2 (*) 157 LOAD_FAST 1 (x) 160 LOAD_FAST 2 (y) 163 CALL_FUNCTION 2 166 LOAD_FAST 3 (more) 169 CALL_FUNCTION 3 172 RETURN_VALUE >> 173 LOAD_CONST 4 (<type 'exceptions.Exception'>) 176 CALL_FUNCTION 0 179 RAISE_VARARGS 1 None Thanks, Timothy Baldridge -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) [View Less]

3 3

Global executable naming policy
by Aaron DeVore Feb. 17, 2012

Feb. 17, 2012

There currently isn't an official naming scheme for the PyPy executable for system installations. Of the 3 setups I've seen, the official build uses pypy, Arch Linux uses pypy, and Gentoo Linux uses pypy-c<version>. I brought up the similar issue of how to differentiate between Python 2 and 3 a while ago, but the conversation sputtered out. One possible naming scheme includes these executables using Python 2.7, Python 3.3, and PyPy 1.9 as an example: pypy pypy-1.9 pypy2 pypy2-1.9 pypy3 … [View More]

4 5

error
by Frans Feb. 14, 2012

Feb. 14, 2012

Hi, After an error report and a succesfull slicing and print yesterday there is another eerror report. I am curious to the fix. -- *Met vriendelijke groeten, kind Regards* * * *Frans*

5 6

Fwd: Discussion with Guido van Rossum and (hopefully) core python-dev on scientific Python and Python3
by Serhat Sevki Dincer Feb. 14, 2012

Feb. 14, 2012

I will take the liberty to forward this topic on "scientific python" to pypy list, since numpy work on pypy has progressed a lot, apparently, and it does concern pypy too, i guess.. Date: Mon, 13 Feb 2012 13:55:45 -0800 From: Fernando Perez <fperez.net(a)gmail.com> Subject: [Matplotlib-users] Discussion with Guido van Rossum and (hopefully) core python-dev on scientific Python and Python3 To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> … [View More]Cc: sage-devel <sage-devel(a)googlegroups.com>, cython-users <cython-users(a)googlegroups.com>, SciPy Users List <scipy-user(a)scipy.org>, networkx-discuss <networkx-discuss(a)googlegroups.com>, enthought-dev <enthought-dev(a)mail.enthought.com>, SciPy Developers List <scipy-dev(a)scipy.org>, Core developer mailing list of the Cython compiler <cython-devel(a)python.org>, matplotlib development list <matplotlib-devel(a)lists.sourceforge.net>, IPython Development list <ipython-dev(a)scipy.org>, IPython User list <ipython-user(a)scipy.org>, pystatsmodels(a)googlegroups.com, Matplotlib Users <matplotlib-users(a)lists.sourceforge.net> Message-ID: <CAHAreOrWczOVZ7fR-WazzTSuzXJqiyk-CCotOdhMR53AY0tzJA(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Hi folks, [ I'm broadcasting this widely for maximum reach, but I'd appreciate it if replies can be kept to the *numpy* list, which is sort of the 'base' list for scientific/numerical work. It will make it much easier to organize a coherent set of notes later on. Apology if you're subscribed to all and get it 10 times. ] As part of the PyData workshop (http://pydataworkshop.eventbrite.com) to be held March 2 and 3 at the Mountain View Google offices, we have scheduled a session for an open discussion with Guido van Rossum and hopefully as many core python-dev members who can make it. We wanted to seize the combined opportunity of the PyData workshop bringing a number of 'scipy people' to Google with the timeline for Python 3.3, the first release after the Python language moratorium, being within sight: http://www.python.org/dev/peps/pep-0398. While a number of scientific Python packages are already available for Python 3 (either in released form or in their master git branches), it's fair to say that there hasn't been a major transition of the scientific community to Python3. Since there is no more development being done on the Python2 series, eventually we will all want to find ways to make this transition, and we think that this is an excellent time to engage the core python development team and consider ideas that would make Python3 generally a more appealing language for scientific work. Guido has made it clear that he doesn't speak for the day-to-day development of Python anymore, so we all should be aware that any ideas that come out of this panel will still need to be discussed with python-dev itself via standard mechanisms before anything is implemented. Nonetheless, the opportunity for a solid face-to-face dialog for brainstorming was too good to pass up. The purpose of this email is then to solicit, from all of our community, ideas for this discussion. In a week or so we'll need to summarize the main points brought up here and make a more concrete agenda out of it; I will also post a summary of the meeting afterwards here. Anything is a valid topic, some points just to get the conversation started: - Extra operators/PEP 225. Here's a summary from the last time we went over this, years ago at Scipy 2008: http://mail.scipy.org/pipermail/numpy-discussion/2008-October/038234.html, and the current status of the document we wrote about it is here: file:///home/fperez/www/site/_build/html/py4science/numpy-pep225/numpy-pep225.html. - Improved syntax/support for rationals or decimal literals? While Python now has both decimals (http://docs.python.org/library/decimal.html) and rationals (http://docs.python.org/library/fractions.html), they're quite clunky to use because they require full constructor calls. Guido has mentioned in previous discussions toying with ideas about support for different kinds of numeric literals... - Using the numpy docstring standard python-wide, and thus having python improve the pathetic state of the stdlib's docstrings? This is an area where our community is light years ahead of the standard library, but we'd all benefit from Python itself improving on this front. I'm toying with the idea of giving a lighting talk at PyConn about this, comparing the great, robust culture and tools of good docstrings across the Scipy ecosystem with the sad, sad state of docstrings in the stdlib. It might spur some movement on that front from the stdlib authors, esp. if the core python-dev team realizes the value and benefit it can bring (at relatively low cost, given how most of the information does exist, it's just in the wrong places). But more importantly for us, if there was truly a universal standard for high-quality docstrings across Python projects, building good documentation/help machinery would be a lot easier, as we'd know what to expect and search for (such as rendering them nicely in the ipython notebook, providing high-quality cross-project help search, etc). - Literal syntax for arrays? Sage has been floating a discussion about a literal matrix syntax (https://groups.google.com/forum/#!topic/sage-devel/mzwepqZBHnA). For something like this to go into python in any meaningful way there would have to be core multidimensional arrays in the language, but perhaps it's time to think about a piece of the numpy array itself into Python? This is one of the more 'out there' ideas, but after all, that's the point of a discussion like this, especially considering we'll have both Travis and Guido in one room. - Other syntactic sugar? Sage has "a..b" <=> range(a, b+1), which I actually think is both nice and useful... There's also the question of allowing "a:b:c" notation outside of [], which has come up a few times in conversation over the last few years. Others? - The packaging quagmire? This continues to be a problem, though python3 does have new improvements to distutils. I'm not really up to speed on the situation, to be frank. If we want to bring this up, someone will have to provide a solid reference or volunteer to do it in person. - etc... I'm putting the above just to *start* the discussion, but the real point is for the rest of the community to contribute ideas, so don't be shy. Final note: while I am here commiting to organizing and presenting this at the discussion with Guido (as well as contacting python-dev), I would greatly appreciate help with the task of summarizing this prior to the meeting as I'm pretty badly swamped in the run-in to pydata/pycon. So if anyone is willing to help draft the summary as the date draws closer (we can put it up on a github wiki, gist, whatever), I will be very grateful. I'm sure it will be better than what I'll otherwise do the last night at 2am :) Cheers, f ps - to the obvious question about webcasting the discussion live for remote participation: yes, we looked into it already; no, unfortunately it appears it won't be possible. We'll try to at least have the audio recorded (and possibly video) for posting later on. pps- if you are close to Mountain View and are interested in attending this panel in person, drop me a line at fernando.perez(a)berkeley.edu. We have a few spots available *for this discussion only* on top of the pydata regular attendance (which is long closed, I'm afraid). But we'll need to provide Google with a list of those attendees in advance. Please indicate if you are a core python committer in your email, as we'll give priority for this overflow pool to core python developers (but will otherwise accommodate as many people as Google lets us). [View Less]

2 1

Socket module fails in pypy1.8
by kracekumar ramaraju Feb. 13, 2012

Feb. 13, 2012

Hi kracekumar@python-lover:~/pypy-pypy-2346207d9946/pypy/translator/sandbox$ pypy1.7 pypy_interact.py --tmp=virtualtmp/ ../../../pypy-c ['/bin/pypy-c'] Warning: cannot find your CPU L2 cache size in /proc/cpuinfo Not Implemented: SomeString(no_nul=True) RuntimeError 'import site' failed Python 2.7.2 (2346207d99463f299f09f3e151c9d5fa9158f71b, Feb 11 2012, 23:26:49) [PyPy 1.8.0 with GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something … [View More]

2 1

PyPy 1.8 50% slower than PyPy 1.7 on my test case
by Laurent Vaucher Feb. 13, 2012

Feb. 13, 2012

Hi and first of all, thanks for that great project. Now to my "problem". I'm doing some puzzle-solving, constraint processing with Python and on my particular program, PyPy 1.8 showed a 50% increase in running time over PyPy 1.7. I'm not doing anything fancy (no numpy, itertools, etc.). The program is here: https://github.com/slowfrog/hexiom To reproduce: - fetch hexiom2.py and level36.txt from github - run 'pypy hexiom2.py -sfirst level36.txt' On my machine (Windows 7) the timings are the … [View More]

1 0

replacing/modifying __import__
by Timothy Baldridge Feb. 13, 2012

Feb. 13, 2012

I'm in the process of writing a Clojure->Python bytecode compiler (http://github.com/halgari/clojure-py). The project is going quite well and runs great on PyPy and CPython. However there is one feature I'm not quite sure how to implement in PyPy. What I would like, is to extend 'import' so that it can handle .clj files: import clojure.core # where there is a file called clojure/core.clj Now in CPython I can simply ovewrite __builtins__.__import__ with a new function that adds additional … [View More]

3 2

How to add 3rd part libraries to sanboxed environment
by kracekumar ramaraju Feb. 13, 2012

Feb. 13, 2012

Hi I am running pypy-1.8 sandboxed one, i created file named `name.py` in current working directory and importing `name.py`, as expected it should fail since sandboxed environment runs on its own file system. sys.path yields ['', '/bin/lib_pypy/__extensions__', '/bin/lib_pypy', '/bin/lib-python/modified-2.7', '/bin/lib-python/2.7', '/bin/lib-python/modified-2.7/lib-tk', '/bin/lib-python/2.7/lib-tk', '/bin/lib-python/2.7/plat-linux2'], os.getcwd() yields /tmp, so how can I install third … [View More]

2 1