Mailman 3 June 2006 - pypy-dev

Security ideas
by Armin Rigo 18 Jul '06

18 Jul '06

Hi all, On Monday I was at an inspiring seminar about (a specific form of) language-level security. I've collected the PyPy-ification of these ideas there: http://codespeak.net/svn/pypy/dist/pypy/doc/discussion/security-ideas.txt Although the focus is different, it makes me think that we could also use similar ideas to implement a form of 'rexec' (restricted execution), with functions compiled by secure() as in the draft above, but running at a priviledge level which is lower than the default ambiant level instead of higher. A bientot, Armin

6 7

Re: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test
by Samuele Pedroni 01 Jul '06

01 Jul '06

antocuni(a)codespeak.net wrote: > Author: antocuni > Date: Fri Jun 30 16:08:16 2006 > New Revision: 29534 > > Modified: > pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > Log: > Added a failing test. > > Basically, if we modify the Record after its hash has already been > computed, we might obtain two objects that compare equal but hash > differently. > > Disabling the caching in LowLevelType.__hash__ fix the test, but I > think it's not what we want. > > Adding the following __hash__ to ootype.Record also fix the test, but > I'm not sure if it's correct, especially with recursive structures: > > def __hash__(self): > return hash(self._fields) > Records are really like lltypesystem Structs, although because some of their code was copied from Instance, they may give the impression that you can add fields after the fact, but that should not be done, it breaks the fact that they are supposed to compare by structure. If you are needing that you need some other approach or introduce some kind of forward definition. _add_fields should really disappear from Record. > > > > Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > ============================================================================== > --- pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py (original) > +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py Fri Jun 30 16:08:16 2006 > @@ -38,3 +38,14 @@ > t.a = 1 > t.b = 2 > assert ooidentityhash(t) != ooidentityhash(t2) > + > +import py > +def test_hash(): > + #py.test.skip("LowLevelType.__hash__ bug waiting to be fixed") > + T1 = Record({"item0": Signed, "item1": Signed}) > + T2 = Record({"item0": Signed}) > + > + hash(T2) # compute the hash, it will stored in __cached_hash > + T2._add_fields({"item1": Signed}) # modify the object > + assert T1 == T2 > + assert hash(T1) == hash(T2) > _______________________________________________ > pypy-svn mailing list > pypy-svn(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn

3 3

patch for __getattribute__()
by Richard Emslie 01 Jul '06

01 Jul '06

Hi! The following patch initiated by a chat with Runar Petursson about descriptors - increases pypy-c/pypy-llvm by 25% on richards.py. I couldnt be bothered creating a branch - but if someone wants to sanity check it and fix the method repr and check it in, you are more than welcome! :-) Cheers, Richard

2 1

Release of PyPy videos
by Wanja Saatkamp 29 Jun '06

29 Jun '06

The PyPy team is happy to announce that the first bunch of PyPy videos can now be downloaded from: http://codespeak.net/pypy/dist/pypy/doc/video-index.html The videos introduce involved people and contain different talks, tutorials and interviews such as: 1. Technical talk on the PyPy architecture, standard interpreter, translation toolchain and Just-in-time compiler at the University of Palma de Mallorca (72 min) 2. Coding discussion between Armin Rigo and Samuele Pedroni during the PyPy sprint at the University of Palma de Mallorca (40 min) 3. Core developer Holger Krekel and project manager Beatrice During explain the agile open source methods used in the PyPy project at Pycon, Dallas (26 min) 4. Core developers Michael Hudson and Christian Tismer are giving an introductory talk at Pycon, Dallas (28 min) 5. A PyPy architecture session given by core developers Holger Krekel and Armin Rigo at Pycon, Dallas (48 min) 6. A Sprint tutorial by core developer Michael Hudson who provides a detailed and hands-on overview about the architecture of PyPy, especially the translation toolchain (44 min) 7. What do you think about PyPy? Interview with American software developer Bob Ippolito at this year's PyCon, Dallas (8 min) 8. Interview with CPython core developer Tim Peters at this year's PyCon, Dallas (23 min) Some technical details: All films are in European PAL system, encoded with DivX and packed in .avi file format. The files can be downloaded using a bittorrent client. Further details regarding downloading and viewing can be found at the URL above. Have fun and don't hesitate to contact us at pypy-dev(a)codespeak.net if you encounter any problems or have any other comments, Best regards, Wanja Saatkamp & the PyPy team

1 0

Re: [pypy-dev] pypy-0.9.0: stackless, new extension compiler
by Armin Rigo 27 Jun '06

27 Jun '06

Hi all, (message by cfbolz which I'm forwarding here) _______________________________________________ Unfortunately the download links for the release tarballs did not work until very recently. They are now working though. You can download the 0.9 release of PyPy under: http://codespeak.net/download/pypy/pypy-0.9.0.tar.bz2 http://codespeak.net/download/pypy/pypy-0.9.0.tar.gz http://codespeak.net/download/pypy/pypy-0.9.0.zip For detailed notes about how to get started into the world of PyPy see here: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html Sorry for the fuss and cheers, Carl Friedrich Bolz _______________________________________________

2 1

Re: [pypy-dev] [Python-Dev] pypy-0.9.0: stackless, new extension compiler
by Ben.Young＠risk.sungard.com 26 Jun '06

26 Jun '06

Congratulations! python-dev-bounces+python=theyoungfamily.co.uk(a)python.org wrote on 25/06/2006 13:07:01: > The PyPy development team has been busy working and we've now packaged > our latest improvements, completed work and new experiments as > version 0.9.0, our fourth public release. > > The highlights of this fourth release of PyPy are: > > **implementation of "stackless" features** > We now support the larger part of the interface of the original > Stackless Python -- see http://www.stackless.com for more. A > significant part of this is the pickling and unpickling of a running > tasklet. > > These features, especially the pickling, can be considered to be a > "technology preview" -- they work, but for example the error handling > is a little patchy in places. > > **ext-compiler** > The "extension compiler" is a new way of writing a C extension for > CPython and PyPy at the same time. For more information, see its > documentation: http://codespeak.net/pypy/dist/pypy/doc/extcompiler.html > > **rctypes** > Most useful in combination with the ext-compiler is the fact that our > translation framework can translate code that uses the > standard-in-Python-2.5 ctypes module. See its documentation for more: > http://codespeak.net/pypy/dist/pypy/doc/rctypes.html > > **framework GCs** > PyPy's interpreter can now be compiled to use a garbage collector > written in RPython. This added control over PyPy's execution makes the > implementation of new and interesting features possible, apart from > being a significant achievement in its own right. > > **__del__/weakref/__subclasses__** > The PyPy interpreter's compatibility with CPython continues improves: > now we support __del__ methods, the __subclasses__ method on types and > weak references. We now pass around 95% of CPython's core tests. > > **logic space preview** > This release contains the first version of the logic object space, > which will add logical variables to Python. See its docs for more: > http://codespeak.net/pypy/dist/pypy/doc/howto-logicobjspace-0.9.html > > **high level backends preview** > This release contains the first versions of new backends targeting high > level languages such as Squeak and .NET/CLI and updated versions of the > JavaScript and Common Lisp backends. They can't compile the PyPy > interpreter yet, but they're getting there... > > **bugfixes, better performance** > As you would expect, performance continues to improve and bugs continue > to be fixed. The performance of the translated PyPy interpreter is > 2.5-3x times faster than 0.8 (on richards and pystone), and is now > stable enough to be able to run CPython's test suite to the end. > > **testing refinements** > py.test, our testing tool, now has preliminary support for doctests. > We now run all our tests every night, and you can see the summary at: > http://snake.cs.uni-duesseldorf.de/pypytest/summary.html > > What is PyPy (about)? > ------------------------------------------------ > > PyPy is a MIT-licensed research-oriented reimplementation of Python > written in Python itself, flexible and easy to experiment with. It > translates itself to lower level languages. Our goals are to target a > large variety of platforms, small and large, by providing a > compilation toolsuite that can produce custom Python versions. > Platform, memory and threading models are to become aspects of the > translation process - as opposed to encoding low level details into > the language implementation itself. Eventually, dynamic optimization > techniques - implemented as another translation aspect - should become > robust against language changes. > > Note that PyPy is mainly a research and development project and does > not by itself focus on getting a production-ready Python > implementation although we do hope and expect it to become a viable > contender in that area sometime next year. > > PyPy is partially funded as a research project under the European > Union's IST programme. > > Where to start? > ----------------------------- > > Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting- > started.html > > PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ > > PyPy Homepage: http://codespeak.net/pypy/ > > The interpreter and object model implementations shipped with the 0.9 > version can run on their own and implement the core language features > of Python as of CPython 2.4. However, we still do not recommend using > PyPy for anything else than for education, playing or research > purposes. > > Ongoing work and near term goals > --------------------------------- > > The Just-in-Time compiler and other performance improvements will be one of > the main topics of the next few months' work, along with finishing the > logic object space. > > Project Details > --------------- > > PyPy has been developed during approximately 20 coding sprints across > Europe and the US. It continues to be a very dynamically and > incrementally evolving project with many of these one-week workshops > to follow. > > PyPy has been a community effort from the start and it would > not have got that far without the coding and feedback support > from numerous people. Please feel free to give feedback and > raise questions. > > contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html > > have fun, > > the pypy team, (Armin Rigo, Samuele Pedroni, > Holger Krekel, Christian Tismer, > Carl Friedrich Bolz, Michael Hudson, > and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) > > PyPy development and activities happen as an open source project > and with the support of a consortium partially funded by a two > year European Union IST research grant. The full partners of that > consortium are: > > Heinrich-Heine University (Germany), AB Strakt (Sweden) > merlinux GmbH (Germany), tismerysoft GmbH (Germany) > Logilab Paris (France), DFKI GmbH (Germany) > ChangeMaker (Sweden), Impara (Germany) > > -- > And not only in the sense that they imagine heretics where these > do not exist, but also that inquistors repress the heretical > putrefaction so vehemently that many are driven to share in it, > in their hatred of the judges. -- The Name Of The Rose, Umberto Eco > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/python%40theyoungfamily.co.uk >

1 0

pypy-0.9.0: stackless, new extension compiler
by Michael Hudson 25 Jun '06

25 Jun '06

The PyPy development team has been busy working and we've now packaged our latest improvements, completed work and new experiments as version 0.9.0, our fourth public release. The highlights of this fourth release of PyPy are: **implementation of "stackless" features** We now support the larger part of the interface of the original Stackless Python -- see http://www.stackless.com for more. A significant part of this is the pickling and unpickling of a running tasklet. These features, especially the pickling, can be considered to be a "technology preview" -- they work, but for example the error handling is a little patchy in places. **ext-compiler** The "extension compiler" is a new way of writing a C extension for CPython and PyPy at the same time. For more information, see its documentation: http://codespeak.net/pypy/dist/pypy/doc/extcompiler.html **rctypes** Most useful in combination with the ext-compiler is the fact that our translation framework can translate code that uses the standard-in-Python-2.5 ctypes module. See its documentation for more: http://codespeak.net/pypy/dist/pypy/doc/rctypes.html **framework GCs** PyPy's interpreter can now be compiled to use a garbage collector written in RPython. This added control over PyPy's execution makes the implementation of new and interesting features possible, apart from being a significant achievement in its own right. **__del__/weakref/__subclasses__** The PyPy interpreter's compatibility with CPython continues improves: now we support __del__ methods, the __subclasses__ method on types and weak references. We now pass around 95% of CPython's core tests. **logic space preview** This release contains the first version of the logic object space, which will add logical variables to Python. See its docs for more: http://codespeak.net/pypy/dist/pypy/doc/howto-logicobjspace-0.9.html **high level backends preview** This release contains the first versions of new backends targeting high level languages such as Squeak and .NET/CLI and updated versions of the JavaScript and Common Lisp backends. They can't compile the PyPy interpreter yet, but they're getting there... **bugfixes, better performance** As you would expect, performance continues to improve and bugs continue to be fixed. The performance of the translated PyPy interpreter is 2.5-3x times faster than 0.8 (on richards and pystone), and is now stable enough to be able to run CPython's test suite to the end. **testing refinements** py.test, our testing tool, now has preliminary support for doctests. We now run all our tests every night, and you can see the summary at: http://snake.cs.uni-duesseldorf.de/pypytest/summary.html What is PyPy (about)? ------------------------------------------------ PyPy is a MIT-licensed research-oriented reimplementation of Python written in Python itself, flexible and easy to experiment with. It translates itself to lower level languages. Our goals are to target a large variety of platforms, small and large, by providing a compilation toolsuite that can produce custom Python versions. Platform, memory and threading models are to become aspects of the translation process - as opposed to encoding low level details into the language implementation itself. Eventually, dynamic optimization techniques - implemented as another translation aspect - should become robust against language changes. Note that PyPy is mainly a research and development project and does not by itself focus on getting a production-ready Python implementation although we do hope and expect it to become a viable contender in that area sometime next year. PyPy is partially funded as a research project under the European Union's IST programme. Where to start? ----------------------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ The interpreter and object model implementations shipped with the 0.9 version can run on their own and implement the core language features of Python as of CPython 2.4. However, we still do not recommend using PyPy for anything else than for education, playing or research purposes. Ongoing work and near term goals --------------------------------- The Just-in-Time compiler and other performance improvements will be one of the main topics of the next few months' work, along with finishing the logic object space. Project Details --------------- PyPy has been developed during approximately 20 coding sprints across Europe and the US. It continues to be a very dynamically and incrementally evolving project with many of these one-week workshops to follow. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support from numerous people. Please feel free to give feedback and raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html have fun, the pypy team, (Armin Rigo, Samuele Pedroni, Holger Krekel, Christian Tismer, Carl Friedrich Bolz, Michael Hudson, and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a two year European Union IST research grant. The full partners of that consortium are: Heinrich-Heine University (Germany), AB Strakt (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) -- Hiro dicks about with his computer, naturally. Being stranded on a life raft in the Pacific is a perfect venue for a hacker. -- Snow Crash, Neal Stephenson

2 1

Random updates
by Armin Rigo 22 Jun '06

22 Jun '06

Hi all, I did a bit of detective work this evening, and found the following. The reason Eric's cron job is no longer showing results for stackless builds is that we broke old-stackless builds of PyPy by adding resume point operations (for unpickling tasklets). They end up as OP_RESUME_STATE_CREATE() macro calls in genc! Argh. Either with fix by somehow detecting in the _stackless module if we are being compiled with old- or new-stackless. Or we don't care any more about the old-stackless and declare it dead for the purpose of compiling PyPy. I also found out that the test summary page at http://snake.cs.uni-duesseldorf.de/pypytest/summary.html was doing a great job at hiding the fact that the tests no longer ran to the end, and that, for more than two weeks now. Either test_ee in pyllvm was getting a SIGIOT signal (people who ever heard about this signal, raise your hand), or test_wrapping was segfaulting (unreproductably - fun). I still have no clue why test_wrapping occasionally segfaults. In any case I fixed the output by considering abnormal termination as a failure. It shows up now at the beginning of the failures. I also modified the misleading '.', which previously only meant 'this test did not fail in that revision' -- e.g. because they were not run at all :-/ Now, a '.' really means 'known to have passed in that revision'. Otherwise you can get an 's', or a ' ' if the test wasn't run at all. Note that before the 12th of June, the necessary information was not collected by the run, so all non-failures there show up as ' ' even if the test was actually run. A bientot, Armin.

3 8

Ideas for Student Proposal.
by Fabrizio Milo 19 Jun '06

19 Jun '06

Hello to everybody, I am an Italian student who will participate in the euro-pypy-sprint. Looking for a good idea for a proposal, I saw https://codespeak.net/issue/pypy-dev/ and many issues are interesting. like: - https://codespeak.net/issue/pypy-dev/issue87 - https://codespeak.net/issue/pypy-dev/issue111 ( or/ and ) - https://codespeak.net/issue/pypy-dev/issue205 - https://codespeak.net/issue/pypy-dev/issue224 The common subject here is "Improving test machinery". What do you think, can issue87 + issue111 be a good base for a proposal? Is it required to produce some proof-of-concept? Are there other ideas in the community not listed in that page? I am a good programmer ( I am a the end of a Computer Science Master course in Rome ) so don't let me escape from helping this wonderful project :) Fabrizio Milo

2 1

Ladies and gentlemen...
by Antonio Cuni 17 Jun '06

17 Jun '06

... I'm proud to announce that gencli is the first high level backend that can compile and run rpystone and richards :-). Some early benchmarks (I post them here so I'll know where to find them when I need it later :-)): pystone ------- gencli: 177429.668653 rpystones/second genc: 4926108.374384 rpystones/second genc w/o backendopt: 1592356.687898 rpystoned/second richards -------- gencli: 28.652590 ms/iteration genc: 7.431851 ms/iteration genc w/o backendopt: 16.203486 ms/iteration ciao Anto

2 1