Mailman 3 October 2005 - pypy-dev

Re: [Python-Dev] AST branch update
by Armin Rigo Oct. 22, 2005

Oct. 22, 2005

Hi Neal, (CC-ing to pypy-dev, if you don't mind :-) On Thu, Oct 20, 2005 at 06:11:54PM -0700, Neal Norwitz wrote: > I was wondering what's going on with PyPy. How is it going? > Did you make good progress at the last sprint? Going quite fine :-) In the last sprint we added optional Stackless-style stack manipulation in a couple of days, which I suppose shows that the approach basically works. > Any interesting ideas that can be used to improve CPython? That's an interesting question. So far PyPy has produced mostly bug reports for obscure errors and crashes in CPython -- only today, trying the CPython HEAD with the newly merged AST compiler showed 7 of them :-) But we are not into language design, and the implementation is significantly different even at the pure Python level. It's difficult to derive direct conclusions so far. Well, our reference counting version is twice as slow as the one using the Boehm GC, but as far as I know it could be all because we are generating faaaaar too many incref/decref all over the place. Anyway, someone might want to try again to scrap Py_INCREF/DECREF and use the Boehm GC with CPython. I suppose that we will soon try more experiments. Coming to mind: unboxed integers to see if they help -- I doubt it, but if they have a large effect then CPython might consider to use them at least in some speed-critical places. Different locking approaches for threads, though it is unclear how much of what we do there would be practical for CPython. Several implementations for app-level string objects, e.g. a "string slice" one, a "lazily joined substrings" one, etc., which might give interesting speed-ups for some programs. All in all I suspect that it will be some more time before we can go past the current state, which is "being a bit like CPython but not quite as good". A bientot, Armin

2 1

Re: [pypy-svn] r18791 - in pypy/dist/pypy/rpython: . module module/test
by Christian Tismer Oct. 21, 2005

Oct. 21, 2005

> pypy/dist/pypy/rpython/ros.py > Log: > * RPythonic helpers to implement os.listdir(): the C-ish functions > ros.opendir(), readdir(), closedir(). > > * support for external types that are externally-managed pointers > (used here for the DIR* pointer). This means that closedir() > must be called explicitely from RPython! This is just perfect! As you said, the purpose of this crap is to provide the smallest possible amount of C code, and the ros functions, in the lack of existing functions which model this (good, that they don't exist!), are just modelling the dumbness of what the C code needs to do. This is of course not meant as an interface for anybody, but a simple model that explains the C code in terms of Python. when-I-called-it-ros-I-thought-of-rotten-OS - ly y'rs - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

playing with gcc
by Michael Hudson Oct. 21, 2005

Oct. 21, 2005

I spent a while compiling some pypy-c source with different optimization options and running pystone (on an iMac with a 1.8 ghz G5 and 1.5 gigs of ram). Here's a table of results: 0 1 2 3 3p s sp c 0 1.00 3.06 3.08 3.08 3.17 3.12 3.25 45.39 1 0.33 1.00 1.01 1.01 1.04 1.02 1.06 14.82 2 0.32 0.99 1.00 1.00 1.03 1.01 1.05 14.72 3 0.32 0.99 1.00 1.00 1.03 1.01 1.05 14.74 3p 0.32 0.97 0.97 0.97 1.00 0.99 1.02 14.31 s 0.32 0.98 0.99 0.99 1.02 1.00 1.04 14.53 sp 0.31 0.94 0.95 0.95 0.98 0.96 1.00 13.98 c 0.02 0.07 0.07 0.07 0.07 0.07 0.07 1.00 Here's a key: 0 : -O0, aka "no optimisation" 1 : -O1 2 : -O2 3 : -O3 3p: -O3 -fbranch-probabilities (i compiled with -fprofile-arcs and ran pystone to generate the branch info) s : -Os. aka "optimize for size" sp: -Os -fbranch-probabilities c : cpython for reference As you can see, you definitely want *some* optimization but it doesn't make a great deal of difference what level you choose. That said, -Os gives the best results and -fbranch-probabilities gives a ~5% gain. I timed each compilation, but forgot to write the results down anywhere. There weren't vast differences at any rate (they all took ~7 minutes, not too bad). Of course, using -fbranch-probabilities requires you compile everything twice. The machine wasn't totally quiet as I ran these tests, but fairly. Cheers, mwh -- What the semicolon's anxious supporters fret about is the tendency of contemporary writers to use a dash instead of a semicolon and thus precipitate the end of the world. -- Lynne Truss, "Eats, Shoots & Leaves"

1 0

Re: [pypy-svn] r18743 - pypy/dist/pypy/module/stackless
by Christian Tismer Oct. 21, 2005

Oct. 21, 2005

arigo(a)codespeak.net wrote: > Author: arigo > Date: Tue Oct 18 18:18:29 2005 > New Revision: 18743 > > Modified: > pypy/dist/pypy/module/stackless/cstack.py I did not expect this going on alone, and I would have appreciated a bit of discussion, as I think we agreed that more thought is needed. -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

3 6

profiling pypy
by Olivier Dormond Oct. 20, 2005

Oct. 20, 2005

Hello, First, congratulation to all pypyers for the great work! As I've some spare time at work this week, I just tried to profile pypy running pystone interactively. The interactive session I used is below and the pickled result can be found bzip2ed at http://codespeak.net/~odie/profile-results-2005-10-20.pickle.bz2 Any suggestion for improving it is welcome :-) and actually any idea how to uses these information in any usefull way would be nice too :-) [pypy/pypy]$ python Python 2.4.1 (#2, Oct 8 2005, 00:22:59) [GCC 4.0.1 (4.0.1-5mdk for Mandriva Linux release 2006.0)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> # From http://codespeak.net/svn/user/arigo/hack/misc/lsprof/ >>> import lsprof >>> p = lsprof.Profiler() >>> try: ... p.enable(True) ... execfile('bin/py.py') ... except: ... pass ... Loading grammar /home/olivier/devel/python/svn/other/pypy/pypy/interpreter/pyparser/data/Grammar2.4 faking <type 'module'> faking <type 'file'> fake-wrapping interp file <open file '<stdout>', mode 'w' at 0xb7c3a068> fake-wrapping interp file <open file '<stderr>', mode 'w' at 0xb7c3a0b0> fake-wrapping interp file <open file '<stdin>', mode 'r' at 0xb7c3a020> PyPy 0.7.1 in StdObjSpace on top of Python 2.4.1 (startupttime: 5.26 secs) >>>> import test.pystone faking <type 'posix.stat_result'> faking <type 'posix.statvfs_result'> >>>> test.pystone.main(1000) Pystone(1.1) time for 1000 passes = 123.84 This machine benchmarks at 8.07494 pystones/second >>>> >>> p.disable() >>> stats = lsprof.Stats(p.getstats()) >>> stats.freeze() >>> import cPickle as pickle >>> pickle.dump(stats, file('profile-results.pickle','w')) >>> Cheers, Odie

2 1

pypy-sync back again thursday 1pm
by Carl Friedrich Bolz Oct. 19, 2005

Oct. 19, 2005

Hi all, the paris sprint is over and even though not everybody has fully recovered yet, a pypy-sync meeting is coming! ======================================== pypy-sync developer meeting 20th Octover ======================================== Time & location: 1pm (30 minutes) at #pypy-sync Regular Topics ==================== - activity reports (3 prepared lines of info, LAST/NEXT/BLOCKERS). - resolve conflicts/blockers Annotated Topics of the week ================================= Status update and next steps ------------------------------------- Since quite a lot of interesting thing have been going on during the Paris sprint it makes sense to get a common understanding what the status of the different topics is and what the next steps should be. Also it makes sense to distribute the work for the following weeks a bit (at least for people not working on reports). To make this a bit less chaotic it makes sense to split this into the following suptopics: - ootypessytem - stackless integration - l3interpreter - compiler - javascript backend This should be quite a big topic already. If anyone has more wishes/suggestions, please send them to me. If anyone will not be able to attend, please send me your activity report before the meeting. See you all tomorrow, Carl Friedrich

1 0

Performance results on Windows
by Ben.Young＠risk.sungard.com Oct. 19, 2005

Oct. 19, 2005

Here are my results of compiling python on Windows with different options: executable richards pystone size (MB) pypy-c 49900 64.5x 953.5 41.4x 9.586 pypy 48791 63.1x 956.9 41.2x 9.633 python 2.4.1 773 1.0x 39434.2 1.0x 0.004 pypy-c is the standard compiled pypy pypy is compiled from a hand-built project with the following options: /Ox /Og /Ob2 /Oi /G7 Which is full program optimization + inline + P4 optimizations. As you can see, there's not much difference, but this is with refcounting which accounts for most of the runtime. I'll attach the project file incase anyone else wants a go: Cheers, Ben

1 0

Toy language
by Ben.Young＠risk.sungard.com Oct. 19, 2005

Oct. 19, 2005

A little toy language that I am posting in response to a discussion on #pypy (It's basically a semi-stackless self modifying language with a syntax similar to python) Do with it what you will (ignoring it most likely!) Run frameparser.py for an example. You do need ply to get it to run. Cheers, Ben

1 0

PPC patch for windows
by Ben.Young＠risk.sungard.com Oct. 18, 2005

Oct. 18, 2005

Hi Everyone, The test skip code for ppc asm wasn't working properly. Here's a patch that gets rid of the test failures (inline as attachments are problematic): Index: translator/asm/ppcgen/test/test_func_builder.py =================================================================== --- translator/asm/ppcgen/test/test_func_builder.py (revision 18731) +++ translator/asm/ppcgen/test/test_func_builder.py (working copy) @@ -2,6 +2,7 @@ import random, sys, os +import py from pypy.translator.asm.ppcgen.ppc_assembler import MyPPCAssembler from pypy.translator.asm.ppcgen.symbol_lookup import lookup from pypy.translator.asm.ppcgen.func_builder import make_func @@ -10,7 +11,7 @@ class TestFuncBuilderTest(object): def setup_class(cls): - if os.uname()[-1] != 'Power Macintosh': + if not hasattr(os, "uname") or os.uname()[-1] != 'Power Macintosh': py.test.skip("can't test all of ppcgen on non-PPC!") def test_simple(self): Index: translator/asm/ppcgen/test/test_ppc.py =================================================================== --- translator/asm/ppcgen/test/test_ppc.py (revision 18731) +++ translator/asm/ppcgen/test/test_ppc.py (working copy) @@ -1,5 +1,6 @@ import random, sys, os +import py from pypy.translator.asm.ppcgen.ppc_assembler import BasicPPCAssembler, MyPPCAssembler from pypy.translator.asm.ppcgen.symbol_lookup import lookup from pypy.translator.asm.ppcgen.regname import * @@ -18,7 +19,7 @@ class TestAssemble(object): def setup_class(cls): - if os.uname()[-1] != 'Power Macintosh': + if not hasattr(os, "uname") or os.uname()[-1] != 'Power Macintosh': py.test.skip("can't test all of ppcgen on non-PPC!") def test_tuplelength(self):

2 1

Re: [pypy-svn] r18730 - pypy/dist/pypy/translator/js
by Christian Tismer Oct. 18, 2005

Oct. 18, 2005

ericvrp(a)codespeak.net wrote: > Author: ericvrp > Date: Tue Oct 18 11:11:38 2005 > New Revision: 18730 > > Modified: > pypy/dist/pypy/translator/js/arraynode.py > pypy/dist/pypy/translator/js/funcnode.py > pypy/dist/pypy/translator/js/node.py > pypy/dist/pypy/translator/js/structnode.py > Log: > Removed slots and some caching. > I don't want to see that kind of code before things really start working! Hi Eric, as a small hint about slots vs. no slots: Instead of removing slots, you can easily make them optional with a global flag. Samuele did that, too. ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

2 1