Mailman 3 December 2010 - pypy-dev

Re: [pypy-dev] Fatal RPython error: AssertionError
by Arnd Rechenburg 22 Dec '10

22 Dec '10

Hi again, It was my mistake. The problem was in 1.4.0 and is solved in 1.4.1 Regards, Arnd ________________________________ From: Arnd Rechenburg Sent: Mittwoch, 22. Dezember 2010 18:00 To: 'pypy-dev(a)codespeak.net' Subject: Fatal RPython error: AssertionError Hi, I have the following problem: Python 2.5.2 (e503e483e9ac, Dec 21 2010, 12:02:48) [PyPy 1.4.1] on linux2 RPython traceback: File "implement_54.c", line 21229, in send_bridge_to_backend File "implement_61.c", line 14547, in Assembler386_assemble_bridge File "implement_63.c", line 44979, in RegAlloc_prepare_bridge File "implement_64.c", line 15487, in RegAlloc__update_bindings File "implement_38.c", line 60625, in RegisterManager__check_invariants Fatal RPython error: AssertionError Any help would be appreciated. Regards, Arnd

1 0

Fatal RPython error: AssertionError
by Arnd Rechenburg 22 Dec '10

22 Dec '10

Hi, I have the following problem: Python 2.5.2 (e503e483e9ac, Dec 21 2010, 12:02:48) [PyPy 1.4.1] on linux2 RPython traceback: File "implement_54.c", line 21229, in send_bridge_to_backend File "implement_61.c", line 14547, in Assembler386_assemble_bridge File "implement_63.c", line 44979, in RegAlloc_prepare_bridge File "implement_64.c", line 15487, in RegAlloc__update_bindings File "implement_38.c", line 60625, in RegisterManager__check_invariants Fatal RPython error: AssertionError Any help would be appreciated. Regards, Arnd

1 0

Re: [pypy-dev] funding/popularity?
by Laura Creighton 22 Dec '10

22 Dec '10

In a message of Wed, 22 Dec 2010 09:24:28 +0100, Miquel Torres writes: >Btw., instead of continuing here "polluting" the pypy-dev mailing >list, we can move to http://groups.google.com/group/codespeed if >needed. I'd rather it stayed here, myself. Laura

1 0

Re: [pypy-dev] Idea for speed.pypy.org
by Laura Creighton 22 Dec '10

22 Dec '10

re: keeping the 'why we are slower/what we could do to fix it' info up to date -- one possibility is to make a 'why we were/what we are for release 1.4.' Then every time you make a major release, you update those fields as needed. And if major changes happen between 'what was in the last major release' vs 'what's on trunk now' you can even make a note of it -- 'fixed in rev whatever it was, when we merged in whoever it was' brank, see blog post over here'. And if you forget, well, you will catch it when the next major release comes out. Just an idea. Laura

10 23

Re: [pypy-dev] Fwd: [IronPython] SciPy
by Gary Robinson 22 Dec '10

22 Dec '10

> I thought this email could be relevant for those interested in SciPy / Numpy > on pypy. With enthought implementing a smaller core and using compatibility > layers for alternative platforms, it would seem to be a good basis for a > pypy port. All I can do is give a BIG +1 to anything that can get numpy/scipy up more quickly. PyPy is starting to give us the speed we need for statistical/scientific work on python (the speed it still lacks compared to C or Java is made up for, for my purposes, by the ease of writing python compared to those languages). The recent 64-bit functionality lets us process a lot of data. The fast-forward branch is giving us the multiprocessing we need. (I recognize that there are other solutions, but for simple things we need to write quickly, the multiprocessing module is really sweet.) The main thing missing now is numpy/scipy. The addition of that will make PyPy a huge win in the scientific community, IMO. Anyway, I mention it in case the opinion of one person who is using Python in production for statistical processing is of interest. -- Gary Robinson CTO Emergent Discovery, LLC personal email: garyrob(a)me.com work email: grobinson(a)emergentdiscovery.com Company: http://www.emergentdiscovery.com Blog: http://www.garyrobinson.net

2 1

[PATCH] Improving readability of generated .c code
by David Malcolm 21 Dec '10

21 Dec '10

The attached patch is an attempt at extending the C code generator so that it annotates the generated .c code with file/line information for the corresponding Python sources. Method: I followed the instructions in pypy/doc/getting-started-dev.txt: cd pypy python bin/translatorshell.py >>> t = Translation(snippet.is_perfect_number) >>> t.annotate([int]) >>> f = t.compile_c() then examine the generated .c code. With this patch, the top of a generated .c function contains an inline copy of the python source, like this: bool_t pypy_g_ll_issubclass__object_vtablePtr_object_vtablePtr(struct pypy_object_vtable0 *l_subcls_0, struct pypy_object_vtable0 *l_cls_0) { bool_t l_v11; long l_v12; long l_v13; long l_v14; /* Python source '/home/david/coding/pypy-svn-anon/trunk-clean/pypy/rpython/lltypesystem/rclass.py' * 665 : def ll_issubclass(subcls, cls): * 666 : return llop.int_between(Bool, cls.subclassrange_min, * 667 : subcls.subclassrange_min, * 668 : cls.subclassrange_max) */ goto block0; ...and for every operation that has an offset, it calculates the linenumber (using the "dis" module on the code object). For every operation that changes the line number, it emits a comment containing that Python source, so that you get code like this: block0: /* 666 : return llop.int_between(Bool, cls.subclassrange_min, */ l_v12 = RPyField(l_cls_0, ov_subclassrange_min); /* 667 : subcls.subclassrange_min, */ l_v13 = RPyField(l_subcls_0, ov_subclassrange_min); /* 668 : cls.subclassrange_max) */ l_v14 = RPyField(l_cls_0, ov_subclassrange_max); OP_INT_BETWEEN(l_v12, l_v13, l_v14, l_v11); goto block1; Hopefully this makes the generated .c much more readable. (I hope to work on the names of locals also; ideally they ought to embed the python indentifier) This is on a Fedora 13 x86_64 box, with cpython 2.6.5 Caveats: - this is my first patch to PyPy (please be gentle!): I hope I've correctly understood the various levels, but I suspect I may still be missing things at the conceptual level - haven't completed the full unit tests yet. - the patch attempts to propagate the "offset" of flowspace SpaceOperation instances in a couple of places where the offset was being discarded: - in BaseInliner.copy_operation - when generating operations in rtyper: set each of the new operations' "offset" to that of the high-level operation - I added the offset to SpaceOperation.__repr__() to help track the above down. - I also pass the reference to the func object from the flowspace FunctionGraph to the rtyper's copy (I wonder if this affect memory usage during translation?) - most functions appear correct, but there are some where I'm not at all convinced by the output (see below) - the patch is assuming that within any given generated C function, there is a single python function, and that the offsets are indexes into the bytecode for that function's code object. I suspect that this is an oversimplification, and why I'm seeing the errors that I'm seeing. - I haven't stripped some of my debugging python comments from funcgen.py in the patch (this is a work in progress). - I haven't tried doing this on a full build of the interpreter (having highly readable generated .c for this is my objective [1]). One other idea I had was to sort the blocks in the function by the bytecode offset; currently the c blocks seem to "jump around" a fair bit relative to the corresponding rpython code. Has any tuning been done on the ordering of the blocks within the generated .c code? (or is it assumed that the .c compiler will "flatten" these, for the case when a node has a single successor node?) (I wonder if similar work could be done on the JVM and CLI code generators?) As mentioned above, I think my patch is getting it wrong in a few places. Here's an example: struct pypy_rpy_string0 *pypy_g_mallocstr__Signed(long l_length_3) { ...snip... /* Python source '/home/david/coding/pypy-svn-anon/trunk-clean/pypy/rpython/lltypesystem/rstr.py' * 35 : def mallocstr(length): * 36 : ll_assert(length >= 0, "negative string length") * 37 : r = malloc(TP, length) * 38 : if not we_are_translated() or not malloc_zero_filled: * 39 : r.hash = 0 * 40 : return r */ ...snip... block15: /* 37 : r = malloc(TP, length) */ OP_ADR_SUB(l_v202, 0, l_v234); l_v235 = (struct pypy_header0 *)l_v234; l_v236 = RPyField(l_v235, h_refcount); /* 38 : if not we_are_translated() or not malloc_zero_filled: */ OP_INT_ADD(l_v236, 1L, l_v237); RPyField(l_v235, h_refcount) = l_v237; goto block5; The C code doesn't look anything like the .py code that my patch is inserting in the above. (My current guess is that I need to be smarter about inlined operations, though that's a guess at this stage) Hope this is helpful Dave [1] http://fedoraproject.org/wiki/Features/PyPyStack is what I'm aiming at

4 4

PyPy 1.4.1
by Armin Rigo 21 Dec '10

21 Dec '10

Hi all, Here is PyPy 1.4.1 :-) Note that the Windows and the Mac OS X 64 builds are still missing; we are waiting for contributions here (amaury, mvt? :-). A bientôt, Armin. =============================== PyPy 1.4.1 =============================== We're pleased to announce the 1.4.1 release of PyPy. This release consolidates all the bug fixes that occurred since the previous release. To everyone that took the trouble to report them, we want to say thank you. http://pypy.org/download.html What is PyPy ============ PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython. Note that it still only emulates Python 2.5 by default; the ``fast-forward`` branch with Python 2.7 support is slowly getting ready but will only be integrated in the next release. In two words, the advantage of trying out PyPy instead of CPython (the default implementation of Python) is, for now, the performance. Not all programs are faster in PyPy, but we are confident that any CPU-intensive task will be much faster, at least if it runs for long enough (the JIT has a slow warm-up phase, which can take several seconds or even one minute on the largest programs). Note again that we do support compiling and using C extension modules from CPython (``pypy setup.py install``). However, this is still an alpha feature, and the most complex modules typically fail for various reasons; others work (e.g. ``PIL``) but take a serious performance hit. Also, for Mac OS X see below. Please note also that PyPy's performance was optimized almost exclusively on Linux. It seems from some reports that on Windows as well as Mac OS X (probably for different reasons) the performance might be lower. We did not investigate much so far. More highlights =============== * We migrated to Mercurial (thanks to Ronny Pfannschmidt and Antonio Cuni) for the effort) and moved to bitbucket. The new command to check out a copy of PyPy is:: hg clone http://bitbucket.org/pypy/pypy * In long-running processes, the assembler generated by old JIT-compilations is now freed. There should be no more leak, however long the process runs. * Improve a lot the performance of the ``binascii`` module, and of ``hashlib.md5`` and ``hashlib.sha``. * Made sys.setrecursionlimit() a no-op. Instead, we rely purely on the built-in stack overflow detection mechanism, which also gives you a RuntimeError -- just not at some exact recursion level. * Fix argument processing (now e.g. ``pypy -OScpass`` works like it does on CPython --- if you have a clue what it does there ``:-)`` ) * cpyext on Mac OS X: it still does not seem to work. I get systematically a segfault in dlopen(). Contributions welcome. * Fix two corner cases in the GC (one in minimark, one in asmgcc+JIT). This notably prevented "pypy translate.py -Ojit" from working on Windows, leading to crashes. * Fixed a corner case in the JIT's optimizer, leading to "Fatal RPython error: AssertionError". * Added some missing built-in functions into the 'os' module. * Fix ctypes (it was not propagating keepalive information from c_void_p). Cheers, Armin Rigo, for the rest of the team

1 0

Fwd: [IronPython] SciPy
by Michael Foord 21 Dec '10

21 Dec '10

I thought this email could be relevant for those interested in SciPy / Numpy on pypy. With enthought implementing a smaller core and using compatibility layers for alternative platforms, it would seem to be a good basis for a pypy port. ---------- Forwarded message ---------- From: Jason McCampbell <jmccampbell(a)enthought.com> Date: 20 December 2010 15:13 Subject: Re: [IronPython] SciPy To: Discussion of IronPython <users(a)lists.ironpython.com> Hi Mark, As Dino mentioned we (Enthought) are working on refactoring Numpy into a pure "C" core with CPython and IronPython interface layers. This is largely complete and available at github (https://github.com/numpy/numpy-refactor), though the core layer is largely undocumented thus far. This is the multi-dimensional array. SciPy is in progress and we are updating it to work with the refactored numpy core and to add an IronPython interface. I assume you are looking for IronPython interfaces to SciPy as opposed to a C interface, correct? Regards, Jason On Thu, Dec 16, 2010 at 1:56 PM, Dino Viehland <dinov(a)microsoft.com> wrote: > Enthought has been working on getting numpy/scipy ported over to work w/ > IronPython. I believe numpy is working but I’m not sure of how far along > SciPy is. There’s a separate mailing list for this at: > > > > https://mail.enthought.com/mailman/listinfo/scipy4dotnet > > > > It’s very low traffic – it’s usually just working through issues Enthought > has run into and either workarounds or suggested changes to IronPython. I’d > suggest sending a mail there – they might have something you can try. > > > > > > *From:* users-bounces(a)lists.ironpython.com [mailto: > users-bounces(a)lists.ironpython.com] *On Behalf Of *Mark Senko > *Sent:* Thursday, December 16, 2010 11:49 AM > *To:* users(a)lists.ironpython.com > *Subject:* [IronPython] SciPy > > > > I’ve been searching for the current state of support for “C” based > libraries, specifically SciPy (I’m just looking for a decent numerical > analysis package). The responses I’ve seen on various websites are somewhat > dated. > > What is the latest status, or is there no effort towards accommodating the > C API? Is IronClad still the best option? Any info, suggestions and warnings > would be appreciated before I start to invest a lot of time into installing > and learning these packages. > > > > *Mark Senko* > > Complete Genomics, Inc. > > 2071 Stierlin Court > > Mountain View, CA 94043 > > > > > > > > ____ > > > > The contents of this e-mail and any attachments are confidential and only for > > use by the intended recipient. Any unauthorized use, distribution or copying > > of this message is strictly prohibited. If you are not the intended recipient > > please inform the sender immediately by reply e-mail and delete this message > > from your system. Thank you for your co-operation. > > > _______________________________________________ > Users mailing list > Users(a)lists.ironpython.com > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > > _______________________________________________ Users mailing list Users(a)lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com -- http://www.voidspace.org.uk

1 0

long.__itemsize__
by Arnd Rechenburg 21 Dec '10

21 Dec '10

Hi, In my code I need something like long.__itemsize__ in Python. Any suggestions to do it with pypy? Regards, Arnd

5 8

Re: [pypy-dev] ImportError: No module named _sqlite3
by Arnd Rechenburg 21 Dec '10

21 Dec '10

Hi, I saw my mistake already by myself. Regards, Arnd ________________________________ From: Arnd Rechenburg Sent: Montag, 20. Dezember 2010 19:16 To: 'pypy-dev(a)codespeak.net' Subject: ImportError: No module named _sqlite3 Hi, I tried to use sqlite3 with pypy but I got the following error: >>>> import sqlite Traceback (most recent call last): File "<console>", line 1, in <module> ImportError: No module named sqlite >>>> import sqlite3 Traceback (most recent call last): File "<console>", line 1, in <module> File "sqlite3/__init__.py", line 24, in <module> from dbapi2 import * File "sqlite3/dbapi2.py", line 27, in <module> from _sqlite3 import * ImportError: No module named _sqlite3 Someone knows where to put the _sqlite3.so library? Any help would be appreciated. Thx in advance, Arnd Rechenburg

1 0