Mailman 3 December 2011 - pypy-dev

The Prototyping Join Patterns with stackless.py Slides
by Andrew Francis 22 Dec '11

22 Dec '11

Hi Folks: I recently gave a rather hastily prepared talk at Montreal Python called "How to Solve a Problem like Santa Claus: Prototyping Join Patterns with stackless.py for Stackless Python." One can get the slides at http://wp.me/pdqoq-46 more information is on my blog. http://andrewfr.wordpress.com once again, I demonstrate how stackless.py can be used to prototype concurrency features for Stackless Python. This time the feature is join patterns. I feel I have come up with a good compromise for implementing this powerful concurrency construct without breaking the Stackless Python API (however channel.balance does become unnecessary). More important though, is during the development, I started to come across several powerful high level and low level concurrency concepts that I think would benefit a future Stackless Python. What particularly pleases me is that we have gone, in little over a year, from a Stackless with no select (a common slight of Stackless Python in the Go Mailing list) to a Stackless version that can handily express stuff that would be rather verbose in Go. And I am the first to attempt that I have a half-baked knowledge of the underlying models for Stackless, greenlets, and now continuelets (I need to shore up my knowledge in these areas). All this would not have been possible without PyPy's stackless.py module. stackless.py rocks! In the weeks to come, I am going to finish off the current prototype, write more documentation, write more tests (just to make sure that I am out of the woods) and examples, build a version that works with continuelets and post everything the Stackless Repository. If folks can accept something more raw and in progress, I would be happy to post the code in a bitbucket account. As was posted out by Richard and Christian in previous posts, the API is a particularly weak area. Anyhow, I would greatly appreciate feedback. Feedback helps the prototyping effects! Cheers, Andrew

1 1

Mixed pypy and cpython with multiprocessing
by Andrew Brown 21 Dec '11

21 Dec '11

Hello everyone, I've been playing around with distributed computing with multiprocessing on CPython, and thought I'd see if I could spin up some workers running pypy and have them connect to a server running CPython. Well, it didn't go so well. The client always gets an IOError: bad message length in the answer_challenge() function of connection.py. Traceback (most recent call last): > File "app_main.py", line 51, in run_toplevel > File "client1.py", line 6, in <module> > m.connect() > File > "[...]/pypy-1.7/lib-python/modified-2.7/multiprocessing/managers.py", line > 474, in connect > conn = Client(self._address, authkey=self._authkey) > File > "[...]/pypy-1.7/lib-python/modified-2.7/multiprocessing/connection.py", > line 149, in Client > answer_challenge(c, authkey) > File > "[...]/pypy-1.7/lib-python/modified-2.7/multiprocessing/connection.py", > line 383, in answer_challenge > message = connection.recv_bytes(256) # reject large message > IOError: bad message length You can try the simple example from the documentation: http://docs.python.org/library/multiprocessing.html#using-a-remote-manager to trigger this. Obviously CPython server and CPython client works, but I've also found that pypy server and pypy client works as well. It's only a mixed server and client that triggers this same error. (either way... pypy server + cpython client or cpython server + pypy client) Is this a bug? Or is multiprocessing not supposed to be compatible across implementations? I would think it is, since it's just sockets with pickled data, right? Thanks, Andrew Brown

3 6

very slow ctypes callbacks
by Arman Bostani 19 Dec '11

19 Dec '11

Hi all, I believe this problem has been brought to your attention before. But, perhaps it something that can be added to the bug list. I'm using a proprietary Windows library which unfortunately uses callbacks a lot. I've attached a simple test case. Under pypy 1.7, this runs about 8 times slower than CPython 2.7. Best, -arman

2 2

OSCON deadline for proposals Jan 12.
by Laura Creighton 18 Dec '11

18 Dec '11

------- Forwarded Message To: conferences(a)python.org From: Aahz <aahz(a)pythoncraft.com> DEADLINE Thursday January 12 OSCON (O'Reilly Open Source Convention), the premier Open Source gathering, will be held in Portland, OR July 16-20. We're looking for people to deliver tutorials and shorter presentations. http://www.oscon.com/oscon2012 http://www.oscon.com/oscon2012/public/cfp/197 Hope to see you there! - -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ ------- End of Forwarded Message

3 2

New sandbox failure: pypy__float2longlong
by Ned Batchelder 16 Dec '11

16 Dec '11

I updated my sandbox work to yesterday's tip, and now I cannot import a number of stdlib modules that used to work fine: ~/pypy> ./pypy/translator/sandbox/pypy_interact.py -q --heapsize=64m pypy-c Warning: cannot find your CPU L2 cache size in /proc/cpuinfo 'import site' failed Python 2.7.1 (7d73e99929bb, Dec 14 2011, 02:24:28) [PyPy 1.7.1-dev0 with GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``Du wirst eben genau das erreichen, woran keiner glaubt'' >>>> import struct Not Implemented: sandboxing for external function 'pypy__float2longlong' Traceback (most recent call last): File "<console>", line 1, in <module> RuntimeError >>>> The modules I've found that cannot be imported are copy, random, and struct. Of course many other modules use those, so lots of stuff has stopped working (json being the important one in my case). Can someone give me clues? (PS: the -q switch to pypy_interact is in my nedbat-sandbox-2 branch, it just suppresses all the logging.) --Ned.

2 1

ctypes tests
by Armin Rigo 14 Dec '11

14 Dec '11

Hi all, I think we could decide by now what to do with the remaining 34 @xfail tests of ctypes. It seems nobody noticed that we don't really implement the full CPython-2.7-compatible ctypes. (As far as I'm concerned, it adds "features" mostly in the form of more and more obscure special cases.) So what do you say about being fine with the current 2.5-compatible situation and silencing the failure on buildbot? http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy-… (Note that there are two failures in the link above. We never noticed when a second failure appeared in addition to the classical xfail one. If anything, it shows that the current situation is bogus.) A bientôt, Armin.

4 8

Free BSD pypy port
by Maciej Fijalkowski 14 Dec '11

14 Dec '11

Hi David, CC pypy-dev Can you explain the choice of options here: http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/pypy/Makefile?rev=1.1;cont… Especially, the objspace, gc and gcrootfinder choices. Cheers, fijal

3 9

No pypy Tkinter demo hello world here ...
by Bengt Richter 14 Dec '11

14 Dec '11

Thought JFTHOI I would try pypy on the simplest demo: [18:05 ~/src/Python-2.7.2/Demo/tkinter/matt]$ cat -n 00-HELLO-WORLD.py 1 from Tkinter import * 2 3 # note that there is no explicit call to start Tk. 4 # Tkinter is smart enough to start the system if it's not already going. 5 6 class Test(Frame): 7 def printit(self): 8 print "hi"py 9 10 def createWidgets(self): 11 self.QUIT = Button(self, text='QUIT', foreground='red', 12 command=self.quit) 13 14 self.QUIT.pack(side=LEFT, fill=BOTH) 15 16 # a hello button 17 self.hi_there = Button(self, text='Hello', 18 command=self.printit) 19 self.hi_there.pack(side=LEFT) 20 21 def __init__(self, master=None): 22 Frame.__init__(self, master) 23 Pack.config(self) 24 self.createWidgets() 25 26 test = Test() 27 test.mainloop() [18:05 ~/src/Python-2.7.2/Demo/tkinter/matt]$ pypy 00-HELLO-WORLD.py pypy: /usr/lib/libcrypto.so.0.9.8: no version information available (required by pypy) pypy: /usr/lib/libssl.so.0.9.8: no version information available (required by pypy) Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "00-HELLO-WORLD.py", line 1, in <module> from Tkinter import * File "/home/bokr/pypy/pypy-c-jit-43780-b590cf6de419-linux/lib-python/2.7/lib-tk/Tkinter.py", line 39, in <module> import _tkinter # If this fails your Python may not be configured for Tk ImportError: No module named _tkinter [18:05 ~/src/Python-2.7.2/Demo/tkinter/matt]$ I guess my "Python may not be configured for Tk" ;-) Is there an easy fix for this? I built my pypy from source, using the simplest config/make/install as specified in the tarball -- IIRC: [18:05 ~/src/Python-2.7.2/Demo/tkinter/matt]$ pypy --version pypy: /usr/lib/libcrypto.so.0.9.8: no version information available (required by pypy) pypy: /usr/lib/libssl.so.0.9.8: no version information available (required by pypy) Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:38) [PyPy 1.5.0-alpha0 with GCC 4.4.3] Also a fix for the crypto / ssl messages, one way or another? Did I just miss some config options? Sorry to be so lazy and just ask ;-/ Decided to send it anyway ... Regards, Bengt Richter

4 3

pypy-1.7 and CLI
by David Naylor 14 Dec '11

14 Dec '11

Hi, Does pypy-1.7 support translating to the CLI backend? On FreeBSD I get the following error: # mono --version Mono JIT compiler version 2.10.6 (tarball Thu Nov 24 17:06:01 UTC 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: amd64 Disabled: none Misc: softdebug LLVM: supported, not enabled. GC: Included Boehm (with typed GC and Parallel Mark) # /usr/local/bin/pypy translate.py --backend=cli -Ojit targetpypystandalone.py <snip/> [translation:ERROR] File "/pypy/translator/cli/query.py", line 25, in get_cli_class [translation:ERROR] return desc.get_cliclass() [translation:ERROR] File "/pypy/translator/cli/query.py", line 169, in get_cliclass [translation:ERROR] BASETYPE = get_ootype(self.BaseType) [translation:ERROR] File "/pypy/translator/cli/query.py", line 91, in get_ootype [translation:ERROR] cliclass = get_cli_class(name) [translation:ERROR] File "/pypy/translator/cli/query.py", line 25, in get_cli_class [translation:ERROR] return desc.get_cliclass() [translation:ERROR] File "/pypy/translator/cli/query.py", line 178, in get_cliclass [translation:ERROR] _static_meth, ootype.StaticMethod) [translation:ERROR] File "/pypy/translator/cli/query.py", line 197, in group_methods [translation:ERROR] res[name] = overload(*meths, **attrs) [translation:ERROR] File "/pypy/translator/cli/dotnet.py", line 242, in __init__ [translation:ERROR] self._resolver = resolver(overloadings) [translation:ERROR] File "/pypy/rpython/ootypesystem/ootype.py", line 1290, in __init__ [translation:ERROR] self._check_overloadings() [translation:ERROR] File "/pypy/rpython/ootypesystem/ootype.py", line 1297, in _check_overloadings [translation:ERROR] raise TypeError, 'Bad overloading' [translation:ERROR] TypeError: Bad overloading Regards, David

4 3

Non standard ASTs
by Timothy Baldridge 13 Dec '11

13 Dec '11

For some time now, I've been working on my Clojure/pypy implementation. However, I'm starting to wish I had a few Clojure facilities while writing my rpython clojure interpreter. Basically I would love to have macros, protocols, symbol resolution, and a ton of other little features for helping with run-time code generation and the like. So this is when I started thinking about generating python code from Clojure source. That is, in Python we have a AST, and in Clojure we can parse LISP lists and convert them to AST constructs. This is when I started to realize where Python falls short of the flexability of Clojure in a few key areas: 1) multiline closures (lambdas with more than one statement)2) IfExpr with multiline bodies. I tried working with the ast module in Python 2.7...and it's horribly restricted. I found a old lib from PEAK that does what I want (BytecodeAssembler) but it's fairly old, and no longer supported. So this is when I thought of the ast in /pypy/interpreter/astcompiler/ Does the pypy interpreter support non-standard constructs like IfExpr with a multiline body? And if not, could it be adapted to support this? I'd rather not drop all the way to emitting Python bytecode...but I may have to. My end goal is to have my target load some rClojure (as I call it) code. And then have pypy translate this rClojure code to a native interpreter. Yes, we're talking about adding a new front-end to pypy. Timothy -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth)

4 5