Mailman 3 August 2011 - pypy-dev

Is PyPy appropriate for Django?
by Andy Aug. 25, 2011

Aug. 25, 2011

Hi, In the PyPy benchmark there's a Django test that shows tremendous speedup of Django when running PyPy-JIT. But that is just for Django templates. What about the other parts of Django? 1) For URL routing Django uses the re module, which is a C extension. Would JIT work with that? 2) Other parts such as DB driver also contains C code, again would that work with JIT? I suppose I could use a pure Python driver, but would a JIT'd pure Python driver be much slower than a native driver? … [View More]

5 9

Removing dead emails from list
by Serhat Sevki Dincer Aug. 25, 2011

Aug. 25, 2011

Whenever I send an email to pypy-dev I get the below bounce back. Can somebody remove that dead email from the list? from postmaster(a)paran.com to jfcgauss(a)gmail.com date Thu, Aug 25, 2011 at 12:18 PM subject [ERR] [pypy-dev] Incompatibility in binary distribution of libraries? mailed-by nrmail7.paran.com hide details 12:18 PM (6 minutes ago) Transmit Report: To: seojiwonn(a)paran.com, 2011/08/25 18:18:18, 452, This mailbox is temporarily disabled. This mailbox is automatically ena

1 0

pypy1.6 slow on string-heavy ops
by Jacob Biesinger Aug. 24, 2011

Aug. 24, 2011

Hi all, New to the list and fairly new to pypy. First of all, congrats on the new 1.6 release-- the growing support for numpy is very exciting (go, fight, win, take state!). So I snagged the 1.6 release to test if it would be faster on the kind of code I often write: Bioinformatics. In this snippet, the point is to check the mappability of the genome-- if a particular substring appears more than once in the genome, the region is called unmappable. Machine Specs: 64bit Ubuntu 11.04 119048 … [View More]

7 13

from the psf board public mailing list.
by Laura Creighton Aug. 24, 2011

Aug. 24, 2011

My comments follow. ------- Forwarded Message Return-Path: psf-board-public-bounces+lac=openend.se(a)python.org Delivery-Date: Wed Aug 24 10:58:13 2011 Steve Holden wrote: > Guido forward this to me, but I think it should be developers getting involved. > Anyway, does anyone on this list have any strong feelings about applying for > funding? I am not sure security is Python's forte, but many people are doing > things with it that I am unaware of. > > regards > Steve … [View More]> Begin forwarded message: >> *From: *Guido van Rossum <guido(a)python.org <mailto:guido@python.org>> >> *Date: *August 23, 2011 4:31:53 PM PDT >> *To: *"Steve Holden, Chairman, PSF" <chairman(a)python.org = >> <mailto:chairman@python.org>> >> *Subject: **Fwd: Get funding for your OSS project from Rapid7* >> >> FYI. I don't know who this is, and he doesn't seem to know me either. :-) >> >> ---------- Forwarded message ---------- >> From: *Christian Kirsch* <Christian_Kirsch(a)rapid7.com = >> <mailto:Christian_Kirsch@rapid7.com>> >> Date: Tue, Aug 23, 2011 at 4:15 PM >> Subject: Get funding for your OSS project from Rapid7 >> To: "guido(a)python.org <mailto:guido@python.org>" <guido(a)python.org> >> Hi Guido van, >> >> __ __ >> >> I hope you're doing great! I was referred to you as the main contact for Python. >> Rapid7 is dedicating $100,000 to support open source projects in the security >> space in 2012 in a program we're calling the Magnificent7. We are looking for >> open source projects that bring value to the information security ecosystem by >> taking an innovative approach to addressing security challenges. Up to seven >> chosen projects will receive funding in 3 phases: an initial outlay, and then >> two further sums tied to the delivery of pre-agreed milestones. >> >> __ __ >> >> Submissions will be judged on several criteria, including a working >> demonstration of the software and the caliber of project roadmap. To be >> eligible, the person participating must have the ability to represent the >> project in question, including updating the web site and doing outbound >> communication on behalf of the project. >> >> After the Rapid7 program committee reviews all initial proposals, promising >> projects will be chosen to participate in the second round of application, >> taking place at the UNITED Security Summit, where they will be granted a = >> 45-minute presentation and in-person Q&A session with the Rapid7 committee. >> Selected recipients for the first funding phase will be announced following >> the UNITED Security Summit and the second round will open for submission= s in >> Q1 2012. >> >> __ __ >> >> To nominate Python for Magnificent7 funding consideration, please visit the >> Rapid7 Community website for details (http://r-7.co/qjE3WG) and guidelines and >> email the submission to oss(a)rapid7.com <mailto:oss@rapid7.com>. Also see our >> press release on the subject (http://r-7.co/n0bVFG). All first round >> submissions are due by September 9, 2011. >> >> Kind regards, >> >> Chris >> Christian Kirsch >> >> Product Marketing Manager <I chopped a zillion footers here. lac> ------- End of Forwarded Message The United Security Conference is Sept 19 and 20 in San Francisco. http://www.net-security.org/conference.php?id=445 Sept 9 is very close for having a proposal ready, and then somebody would have to go to SF. On the other hand, the sandboxing in PyPy is definitely innovative, and if they are just announcing their contest now, there may not _be_ many other people competing for the money. Of course we would have to find out something to do with sandboxing. Laura [View Less]

3 2

Re: [pypy-dev] 1.6 status report
by Gabriel Lavoie Aug. 23, 2011

Aug. 23, 2011

I'm on 7.4-STABLE. Anyway, as I understand, FreeBSD will never upgrade its core GCC version higher than 4.2.2 due to GPL version switch to GPLv3... Knowing there is an issue with GCC 4.2, I'll finish my testing with the latest GCC available from ports and I'll report. It might be a good idea to make a newer GCC version a requirement in your PyPy port. A better idea would be to create "officials" PyPy build (probably just with the jit default setup, 32bits and 64bits) and make a port that would … [View More]install the binary package instead of translating the source code, since translation has a lot of requirements (including hardware) and takes a lot of time. Gabriel 2011/8/16 David Naylor <naylor.b.david(a)gmail.com> > I was not aware of that. If you are on FreeBSD-current then clang is > also available (and is also in ports). clang however takes much longer > to compile than gcc but can easily be run in parallel whereas gcc > used too much memory to allow that. > > On 16 August 2011 08:10, Gabriel Lavoie <glavoie(a)gmail.com> wrote: > > Okay! It seems I've hit the GCC 4.2 memory usage bug: > > https://bugs.launchpad.net/ubuntu/+source/gcc-4.2/+bug/187391 > > http://comments.gmane.org/gmane.comp.python.pypy/7634 > > > > I'm now trying to build with GCC 4.6 (a port is available) and I'll > report > > back. > > > > Gabriel > > > > 2011/8/16 David Naylor <naylor.b.david(a)gmail.com> > >> > >> On Tuesday, 16 August 2011 05:50:18 Gabriel Lavoie wrote: > >> > Hi David, > >> > I took a look at your port and it looks good! The patch you made > >> > for > >> > SANDBOX should probably not be needed, with a fix upstream. It seems > >> > PyPy > >> > still have a few "hardcoded path" issues that need to be looked at for > >> > FreeBSD (and probably for Darwin too for tools like Darwin Ports). I > >> > felt > >> > on one of those issues this morning with "expat.h" and "libexpat.so". > >> > The > >> > translation process didn't look at /usr/local/include and > /usr/local/lib > >> > to find them. Actually, there are a few places where the > >> > "pypy/translator/platform" module should be use to find include/lib > >> > paths. > >> > I made the initial "freebsd.py" file in there that contains "general" > >> > references to include/lib directories in "/usr/local". > >> > >> Thanks, sandbox has not been tested as I do not know how to. An example > >> script that makes sandbox behave like a normal python/pypy instance > would > >> be > >> nice... > >> > >> On the expat issue: > >> # ldd `which pypy1.5` > >> /usr/local/bin/pypy1.5: > >> libm.so.5 => /lib/libm.so.5 (0x801664000) > >> libintl.so.9 => /usr/local/lib/libintl.so.9 (0x801885000) > >> libssl.so.6 => /usr/lib/libssl.so.6 (0x801a8e000) > >> libcrypto.so.6 => /lib/libcrypto.so.6 (0x801ce1000) > >> libexpat.so.6 => /usr/local/lib/libexpat.so.6 (0x802082000) > >> libbz2.so.4 => /usr/lib/libbz2.so.4 (0x8022a5000) > >> libz.so.6 => /lib/libz.so.6 (0x8024b5000) > >> librt.so.1 => /usr/lib/librt.so.1 (0x8026cb000) > >> libutil.so.9 => /lib/libutil.so.9 (0x8028d0000) > >> libffi.so.5 => /usr/local/lib/libffi.so.5 (0x802ae1000) > >> libncurses.so.8 => /lib/libncurses.so.8 (0x802ce7000) > >> libcrypt.so.5 => /lib/libcrypt.so.5 (0x802f34000) > >> libthr.so.3 => /lib/libthr.so.3 (0x803154000) > >> libc.so.7 => /lib/libc.so.7 (0x803377000) > >> libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x8036bf000) > >> > >> I also see those error yet it appears expat is picked up somehow, maybe > as > >> a > >> depenendency? > >> > >> > About my current issue, maybe you've seen it. When I try to build PyPy > >> > using CPython, I get a "Broken Pipe" error and the process fails > (using > >> > the standard "python translate.py -O2" command). Have you seen this > >> > problem recently? I'm trying with the latest trunk. It seems the > >> > "python" > >> > process runs out of memory as I have this in my kernel log: > >> > >> I have not encountered that problem before (except I suffer from > ENOMEM). > >> > >> > swap_pager_getswapspace(16): failed > >> > pid 15107 (cc1), uid 1001, was killed: out of swap space > >> > swap_pager: out of swap space > >> > swap_pager_getswapspace(2): failed > >> > pid 37981 (python), uid 1001, was killed: out of swap space > >> > >> I have seen this problem before. FreeBSD hard codes the size of swap it > >> can > >> access, and does not adapt it based on RAM or active swap. To fix that > >> try > >> the following in /boot/loader.conf: > >> > >> # Increase max swap zone to allow for ~64GB of swap > >> kern.maxswzone="268435456" # 256MB > >> > >> > PyPy used to build correctly but I haven't tried to build it in the > last > >> > 6 > >> > months... > >> > >> I have successfully built pypy-1.5 and trunk (~1 week ago), so it is > >> possible. > >> If you are still having problems please send me the installed physical > >> memory, > >> and the output of `swapinfo` and `uname -a`. > >> > >> > 2011/8/15 David Naylor <naylor.b.david(a)gmail.com> > >> > > >> > > On Monday, 15 August 2011 17:30:33 Gabriel Lavoie wrote: > >> > > > Hi David, > >> > > > > >> > > > for your information, I'm currently trying to make my FreeBSD > >> > > > >> > > buildbot > >> > > > >> > > > back online. It broke when the requirement for the "pypy" binary > to > >> > > > build PyPy appeared. It seems there are still some issues to make > >> > > > PyPy > >> > > > build again under FreeBSD. I'll keep you updated. > >> > > > >> > > Thanks for the update. I have been working on a port for pypy. The > >> > > code > >> > > is > >> > > at github.com/DragonSA/pypy. It may be able to help you compile > pypy > >> > > under > >> > > FreeBSD. There is a branch called 1.6 that is compatible with > trunk. > >> > > To > >> > > compile from 1.6 set WRKSRC to point to a fresh checkout of pypy: > >> > > > >> > > # make install WRKSRC=/path/to/hg/checkout/of/pypy > >> > > > >> > > The master branch uses the 1.5 sources. > >> > > > >> > > > 2011/8/11 David Naylor <naylor.b.david(a)gmail.com> > >> > > > > >> > > > > On Thursday, 11 August 2011 11:08:18 Maciej Fijalkowski wrote: > >> > > > > > Hi > >> > > > > > > >> > > > > > A little status update from my side. > >> > > > > > > >> > > > > > I'm trying to get 1.6 out of the door and there are a few > things > >> > > > > > that stand out before we can do it. > >> > > > > > > >> > > > > > * Windows situation (a lot of crashes with missing liveness). > >> > > > > > Also > >> > > > > > lib-python tests take 10h which is a bit too much IMO (not a > new > >> > > > > > regression though) > >> > > > > > * OS X situation (missing buildbots, being addressed now) > >> > > > > > * A bunch of issues that either should be decided we don't > care > >> > > > > > or > >> > > > > > we do care and fix it. > >> > > > > > * Missing jitviewer release/demo that I'm doing right now > >> > > > > > * Alex reported sqlite crashes > >> > > > > > * random segfault when running trackgcroot, armin, did you > have > >> > > > > > a > >> > > > > > look? I can do if you don't > >> > > > > > >> > > > > Would anyone be interested in the FreeBSD situation? If so what > >> > > > >> > > support > >> > > > >> > > > > do you need? I am currently updating my repository and will > >> > > > > report > >> > > > > back... > >> > > > > > >> > > > > _______________________________________________ > >> > > > > pypy-dev mailing list > >> > > > > pypy-dev(a)python.org > >> > > > > http://mail.python.org/mailman/listinfo/pypy-dev > > > > > > > > -- > > Gabriel Lavoie > > glavoie(a)gmail.com > > > -- Gabriel Lavoie glavoie(a)gmail.com [View Less]

3 4

test_ppc successful on PPC64
by David Edelsohn Aug. 22, 2011

Aug. 22, 2011

I fixed test_call_function on PPC64 so all test_ppc.py tests succeed on PPC64. Now on to the more general tests. - David

1 0

How to test code in pypy.rpython?
by Mitchell Hashimoto Aug. 22, 2011

Aug. 22, 2011

Hello! I've been working on some 'os' standard library methods, which requires some interp-level code in pypy.rpython.module.ll_os. I noticed that while in the "py.py" interpreter that the methods I've been adding to "os" have been using CPython's library. I've written unit tests which appear to actually test the ll_os code and it seems to work, and I can also always drop back into the interp-level repl and do a "extregistry.lookup(fn).lltypeimpl" trick to get the method and it seems to work … [View More]

3 2

regex (new re) and 1.6
by Serhat Sevki Dincer Aug. 20, 2011

Aug. 20, 2011

http://pypi.python.org/pypi/regex installs nicely on pypy-1.6 on my Ubuntu 11.04 i386 but >>>> from regex import sub Traceback (most recent call last): File "<console>", line 1, in <module> File "/var/Home/serhat/Program/pypy-1.6/site-packages/regex.py", line 343, in <module> from _regex_core import * File "/var/Home/serhat/Program/pypy-1.6/site-packages/_regex_core.py", line 21, in <module> import _regex ImportError: unable to load extension … [View More]

4 4

PyGame for translatorshell on Lion
by Mitchell Hashimoto Aug. 20, 2011

Aug. 20, 2011

Hello, I've been trying to play around with the PyPy source and work on fixing some bugs. One of the things I wanted to look at was the graph files generated by the shell. I installed PyGame and GraphViz and when I run a "t.view" on a Translation object, I get the following crazy thing. Has anyone see this? Is this something I should bring up to the PyGame guys? Does anyone have this working on Lion (assuming yes)? Best, Mitchell Aug 20 00:57:14 picochip.local python[58716] <Error>: … [View More]kCGErrorInvalidConnection: CGSGetCurrentCursorLocation: Invalid connection Aug 20 00:57:14 picochip.local python[58716] <Error>: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged. Aug 20 00:57:14 picochip.local python[58716] <Error>: kCGErrorInvalidConnection: CGSGetCurrentCursorLocation: Invalid connection Aug 20 00:57:14 picochip.local python[58716] <Error>: kCGErrorInvalidConnection: CGSNewWindowWithOpaqueShape: Invalid connection 2011-08-20 00:57:14.130 python[58716:707] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Error (1002) creating CGSWindow on line 262' *** First throw call stack: ( 0 CoreFoundation 0x00007fff81445986 __exceptionPreprocess + 198 1 libobjc.A.dylib 0x00007fff88125d5e objc_exception_throw + 43 2 CoreFoundation 0x00007fff814457ba +[NSException raise:format:arguments:] + 106 3 CoreFoundation 0x00007fff81445744 +[NSException raise:format:] + 116 4 AppKit 0x00007fff8a686e78 _NSCreateWindowWithOpaqueShape2 + 788 5 AppKit 0x00007fff8a5d4c80 -[NSWindow _commonAwake] + 1875 6 AppKit 0x00007fff8a5d3894 -[NSWindow _commonInitFrame:styleMask:backing:defer:] + 1871 7 AppKit 0x00007fff8a5d28e0 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1502 8 AppKit 0x00007fff8a5d22b1 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 47 9 libSDL-1.2.0.dylib 0x00000001081d5cfd -[SDL_QuartzWindow initWithContentRect:styleMask:backing:defer:] + 285 10 libSDL-1.2.0.dylib 0x00000001081d30a4 QZ_SetVideoMode + 1076 11 libSDL-1.2.0.dylib 0x00000001081c9e8a SDL_SetVideoMode + 602 12 display.so 0x000000010824f489 set_mode + 361 13 python 0x0000000107d63bab PyEval_EvalFrameEx + 18011 14 python 0x0000000107d66f43 fast_function + 179 15 python 0x0000000107d63c5d PyEval_EvalFrameEx + 18189 16 python 0x0000000107d66e07 PyEval_EvalCodeEx + 2103 17 python 0x0000000107d66fb8 fast_function + 296 18 python 0x0000000107d63c5d PyEval_EvalFrameEx + 18189 19 python 0x0000000107d66e07 PyEval_EvalCodeEx + 2103 20 python 0x0000000107cf480b function_call + 347 21 python 0x0000000107ccc141 PyObject_Call + 97 22 python 0x0000000107cdddf7 instancemethod_call + 503 23 python 0x0000000107ccc141 PyObject_Call + 97 24 python 0x0000000107d2798a slot_tp_init + 106 25 python 0x0000000107d203a5 type_call + 229 26 python 0x0000000107ccc141 PyObject_Call + 97 27 python 0x0000000107d63e75 PyEval_EvalFrameEx + 18725 28 python 0x0000000107d66f43 fast_function + 179 29 python 0x0000000107d63c5d PyEval_EvalFrameEx + 18189 30 python 0x0000000107d66e07 PyEval_EvalCodeEx + 2103 31 python 0x0000000107cf480b function_call + 347 32 python 0x0000000107ccc141 PyObject_Call + 97 33 python 0x0000000107d6451a PyEval_EvalFrameEx + 20426 34 python 0x0000000107d66f43 fast_function + 179 35 python 0x0000000107d63c5d PyEval_EvalFrameEx + 18189 36 python 0x0000000107d66e07 PyEval_EvalCodeEx + 2103 37 python 0x0000000107d66fb8 fast_function + 296 38 python 0x0000000107d63c5d PyEval_EvalFrameEx + 18189 39 python 0x0000000107d66e07 PyEval_EvalCodeEx + 2103 40 python 0x0000000107d66e86 PyEval_EvalCode + 54 41 python 0x0000000107d851a4 PyRun_FileExFlags + 164 42 python 0x0000000107d86458 PyRun_SimpleFileExFlags + 392 43 python 0x0000000107d99729 Py_Main + 2969 44 python 0x0000000107cc5f84 start + 52 45 ??? 0x0000000000000004 0x0 + 4 ) terminate called throwing an exceptionTraceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/mitchellh/code/3rdparty/python/pypy/pypy/translator/interactive.py", line 36, in view self.context.view() File "/Users/mitchellh/code/3rdparty/python/pypy/pypy/translator/translator.py", line 151, in view FlowGraphPage(self).display() File "/Users/mitchellh/code/3rdparty/python/pypy/dotviewer/graphpage.py", line 32, in display graphclient.display_page(self, save_tmp_file=self.save_tmp_file) File "/Users/mitchellh/code/3rdparty/python/pypy/dotviewer/graphclient.py", line 44, in display_page reload(0) File "/Users/mitchellh/code/3rdparty/python/pypy/dotviewer/graphclient.py", line 40, in reload send_graph_messages(io, messages) File "/Users/mitchellh/code/3rdparty/python/pypy/dotviewer/graphclient.py", line 98, in send_graph_messages raise ioerror IOError: connection unexpectedly closed (graphserver crash?) [View Less]

2 1

PyPy 1.6 released
by Maciej Fijalkowski Aug. 20, 2011

Aug. 20, 2011

======================== PyPy 1.6 - kickass panda ======================== We're pleased to announce the 1.6 release of PyPy. This release brings a lot of bugfixes and performance improvements over 1.5, and improves support for Windows 32bit and OS X 64bit. This version fully implements Python 2.7.1 and has beta level support for loading CPython C extensions. You can download it here: http://pypy.org/download.html What is PyPy? ============= PyPy is a very compliant Python interpreter, … [View More]almost a drop-in replacement for CPython 2.7.1. It's fast (`pypy 1.5 and cpython 2.6.2`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64 or Mac OS X. Windows 32 is beta (it roughly works but a lot of small issues have not been fixed so far). Windows 64 is not yet supported. The main topics of this release are speed and stability: on average on our benchmark suite, PyPy 1.6 is between **20% and 30%** faster than PyPy 1.5, which was already much faster than CPython on our set of benchmarks. The speed improvements have been made possible by optimizing many of the layers which compose PyPy. In particular, we improved: the Garbage Collector, the JIT warmup time, the optimizations performed by the JIT, the quality of the generated machine code and the implementation of our Python interpreter. .. _`pypy 1.5 and cpython 2.6.2`: http://speed.pypy.org Highlights ========== * Numerous performance improvements, overall giving considerable speedups: - better GC behavior when dealing with very large objects and arrays - **fast ctypes:** now calls to ctypes functions are seen and optimized by the JIT, and they are up to 60 times faster than PyPy 1.5 and 10 times faster than CPython - improved generators(1): simple generators now are inlined into the caller loop, making performance up to 3.5 times faster than PyPy 1.5. - improved generators(2): thanks to other optimizations, even generators that are not inlined are between 10% and 20% faster than PyPy 1.5. - faster warmup time for the JIT - JIT support for single floats (e.g., for ``array('f')``) - optimized dictionaries: the internal representation of dictionaries is now dynamically selected depending on the type of stored objects, resulting in faster code and smaller memory footprint. For example, dictionaries whose keys are all strings, or all integers. Other dictionaries are also smaller due to bugfixes. * JitViewer: this is the first official release which includes the JitViewer, a web-based tool which helps you to see which parts of your Python code have been compiled by the JIT, down until the assembler. The `jitviewer`_ 0.1 has already been release and works well with PyPy 1.6. * The CPython extension module API has been improved and now supports many more extensions. For information on which one are supported, please refer to our `compatibility wiki`_. * Multibyte encoding support: this was of of the last areas in which we were still behind CPython, but now we fully support them. * Preliminary support for NumPy: this release includes a preview of a very fast NumPy module integrated with the PyPy JIT. Unfortunately, this does not mean that you can expect to take an existing NumPy program and run it on PyPy, because the module is still unfinished and supports only some of the numpy API. However, barring some details, what works should be blazingly fast :-) * Bugfixes: since the 1.5 release we fixed 53 bugs in our `bug tracker`_, not counting the numerous bugs that were found and reported through other channels than the bug tracker. Cheers, Hakan Ardo, Carl Friedrich Bolz, Laura Creighton, Antonio Cuni, Maciej Fijalkowski, Amaury Forgeot d'Arc, Alex Gaynor, Armin Rigo and the PyPy team .. _`jitviewer`: http://morepypy.blogspot.com/2011/08/visualization-of-jitted-code.html .. _`bug tracker`: https://bugs.pypy.org .. _`compatibility wiki`: https://bitbucket.org/pypy/compatibility/wiki/Home [View Less]

3 6