Mailman 3 February 2003 - pypy-dev

pypy-dev List Summary - 09 Feb 2003 to 16 Feb 2003
by Troy Melhase 18 Feb '03

18 Feb '03

This is a summary of the discussions on the pypy-dev mailing list between 09 February and 16 February 2003. The full archive of the list can be found here: http://codespeak.net/pipermail/pypy-dev/ ZODB and Threads ---------------- In a continuation from last weeks thread discussing multiple VMs in a single process, Jeremy Hylton says multi-threaded ZODB applications can use multiple connections. His response prompts Tobias Oberstein to share his belief that the GIL is the real issue. He suggests a replacement as well: http://codespeak.net/pipermail/pypy-dev/2003q1/000416.html Wiki Content ------------ Armin Rigo posts links to summaries he has created in the wiki: http://codespeak.net/pipermail/pypy-dev/2003q1/000412.html Reusing Parrot -------------- Tobias Oberstein brings up his recent discovery of the Parrot VM: http://codespeak.net/pipermail/pypy-dev/2003q1/000413.html Lalo Martins thinks Parrot isn't relevent because it would only provide a different VM. Christian Tismer concurs that it wouldn't be useful, instead because the VM isn't yet finished: http://codespeak.net/pipermail/pypy-dev/2003q1/000418.html Generating C ------------ Steven Shaw suggests that Minimal Python could generate C code, compile it, and load it at runtime. He cites Goo and GCL as examples: http://codespeak.net/pipermail/pypy-dev/2003q1/000425.html Chris responds by saying C code is already a backend target, and that Psyco would probably be faster: http://codespeak.net/pipermail/pypy-dev/2003q1/000426.html Scons ----- Holger Krekel asks the list readers for opinions on Scons. Chris says he has been quite satisfied with it: http://codespeak.net/pipermail/pypy-dev/2003q1/000422.html Charles Crain, a Scons developer, says Scons is "'technically' in alpha" but extremely stable and well-tested: http://codespeak.net/pipermail/pypy-dev/2003q1/000430.html Bram Moolenaar, lead developer of A-A-P, suggests A-A-P as an alternative and lists the similarities and differences between the two: http://codespeak.net/pipermail/pypy-dev/2003q1/000449.html Sprint Updates -------------- VanL asks if sprint materials can be made available during the sprint. He asks for recordings of the talks, presentation materials, and nightly code snapshots: http://codespeak.net/pipermail/pypy-dev/2003q1/000419.html Chris says daily updates will go in the wiki, that the code will be available, and that Dinu Gherman will probably write an article: http://codespeak.net/pipermail/pypy-dev/2003q1/000420.html Bengt Richter asks for mp3s, and Dinu suggests AV streams: http://codespeak.net/pipermail/pypy-dev/2003q1/000423.html PyPy at OSCON ------------- Laura Creighton asks the list readers if anyone will be presenting or speaking at OSCON: http://codespeak.net/pipermail/pypy-dev/2003q1/000427.html Chris says he's going to PyCON and can't attend both. Holger says he doesn't know if he's going and suggests Laura do the presentation: http://codespeak.net/pipermail/pypy-dev/2003q1/000436.html Laura proposes a report on the PyPy sprint and the project: http://codespeak.net/pipermail/pypy-dev/2003q1/000429.html Object Model ------------ Armin discusses his reorganized thoughts regarding the distinction between "application-level" and "interpreter-level" objects. He now believes in following the lead of the CPython main loop by not assuming anything about the objects upon which it operates. He suggests implementing an 'ObjectSpace' class and making its instances attributes of frame objects. These 'objectspace' attributes would be invisible to the application and would define the "library" needed to perform the operations of the interpreter. The objectspaces would define 'add', 'getitem', and more, in addition to 'wrap' and 'unwrap' methods: http://codespeak.net/pipermail/pypy-dev/2003q1/000431.html Michael Hudson likes the idea, and Armin explains that exceptions need to be caught in the objectspaces: http://codespeak.net/pipermail/pypy-dev/2003q1/000445.html VanL asks for clarification by drawing analogies from his hardware background, and Armin thinks his analogy is interesting and coherent: http://codespeak.net/pipermail/pypy-dev/2003q1/000434.html Samuele Pedroni thinks the freedom provided by this approach also means increased duplication: http://codespeak.net/pipermail/pypy-dev/2003q1/000438.html Armin believes this can be mitigated by a multi-method-dispatch mechanism in an abstract class: http://codespeak.net/pipermail/pypy-dev/2003q1/000447.html Rocco Moretti wonders if 'wrap' and 'unwrap' would be necessary, and Armin makes compelling arguments for them despite the drawbacks. Armin likes the symmetry between 'wrap' and 'unwrap' for a 'default' implementation, and thinks the two methods could be the basis of network objectspaces: http://codespeak.net/pipermail/pypy-dev/2003q1/000448.html Armin follows up to his own post by suggesting type-based dispatch may be useful, and that both approaches should be tried: http://codespeak.net/pipermail/pypy-dev/2003q1/000456.html VanL asks if bytecode semantics are to be preserved with the same rigor as language semantics. He wonders if a register-based VM could replace the stack-based approach. Paolo Invernizzi believes different VM implementations are possible: http://codespeak.net/pipermail/pypy-dev/2003q1/000443.html Memory Management ----------------- Richard Emslie broaches the subject of memory management. He wonders how PyPy will handle bytes at a lower level: http://codespeak.net/pipermail/pypy-dev/2003q1/000450.html Chris responds that the intent is to not reimplement the the CPython GC, as it is overly visible in CPython. He lists the ways it encumbers the C source and describes alternate object-referencing semantics: http://codespeak.net/pipermail/pypy-dev/2003q1/000451.html Holger asks for clarification on the different algorithmic view of INCREF and DECREF, and Armin uses parameter passing as an example: http://codespeak.net/pipermail/pypy-dev/2003q1/000457.html Chris says INCREF and DECREF aren't wanted, but instead a method to increase and decrease the "liveliness" of an object. He thinks one approach may be C-style automatic variables, but reminds that it's only one of many possible implementations: http://codespeak.net/pipermail/pypy-dev/2003q1/000460.html Down Time --------- Holger sends notice that codespeak.net, the PyPy site and lists will be down while the hardware is moved to the sprint location: http://codespeak.net/pipermail/pypy-dev/2003q1/000453.html Subverted --------- Holger announces Subversion as the planned source-control platform for PyPy: http://codespeak.net/pipermail/pypy-dev/2003q1/000454.html Armin points out that Subversion must be downloaded twice to bootstrap itself, and hopes that PyPy will be more efficient: http://codespeak.net/pipermail/pypy-dev/2003q1/000458.html Chris asks if CVS would be a better choice, but Holger thinks Subversion should be tried for the sprint: http://codespeak.net/pipermail/pypy-dev/2003q1/000463.html Laura Creighton adds that Internet Explorer(tm) users can connect to the Tigris SVN: http://codespeak.net/pipermail/pypy-dev/2003q1/000455.html Minimally, -troy

1 0

downtime codespeak/pypy
by holger krekel 17 Feb '03

17 Feb '03

Hello, we plan to transfer the codespeak-computer which hosts the pypy site to the sprint-place. This gives us 100 MBit access to our repository among other conveniencies. So expect some downtimes on early sunday afternoon after which the DNS-switch should be completed. mail to the pypy-dev list should be automatically deferred until the host is online again. Of course it would be nicer to have a second computer which could take-over without downtime but Jens-Uwe didn't manage to get something like this done, yet. regards, holger

2 2

repository access
by holger krekel 16 Feb '03

16 Feb '03

Hi again, we plan to use subversion for managing our code. If you want to participate from the outside or otherwise share/contribute code please try to setup a subversion client. I've heard that installing it is not too obvious from the http://subversion.tigris.org website so if you succeed installing it on your system, success stories are certainly welcome. For people using Gentoo Linux i've have success doing an ebuild on "Neon" and subsequently "subversion" which appears to work. The real hard part is to install the server side but this already works and a test server is reachable at http://codespeak.net:8080/ (subversion roots are urls and WebDav/DeltaX is the client/server network protocol). holger

4 7

Re: [pypy-dev] Object model
by roccomoretti＠netscape.net 16 Feb '03

16 Feb '03

Armin Rigo <arigo(a)tunes.org> wrote: >One of the problems with wrap()/unwrap() is when they are applied to >containers. As they are, they have to "translate" the whole structure >recursively, which is often not a good idea. > >On the other hand, if we define more specialized purpose-oriented methods, we >loose opportunities given by wrap()/unwrap() to let particular routines send >or examine an arbitrary piece of data. I'm thinking about type-based >dispatch, for example, where you have to query the type of an object with the >appropriate ObjectSpace method, and then unwrap() this type. > >I don't know which one is better. Maybe both are useful. I guess we will >have to try and see. Hopefully, trying both should be easy in the early >phases. Now that I think of it, there is is no reason *not* to have both. As much as feasable, we can define special-purpose functions to aid in optimization and separation of concerns (e.g. have the truth() converter), but also have the generic functions (e.g. unwrap()) so that there is maximum flexibility available if we choose to use it. If we allow each to not work for some cases, having both can incorporate either extreme (just have the other function always fail). We can get the best of all worlds. -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/

1 0

Runtime information from tests
by holger krekel 16 Feb '03

16 Feb '03

Hello, the following idea has been circulating in my mind quite a bit. I'll put this to discussion on the sprint but you may want to comment here on the list, too. I think that we should generally use our (unit-) tests in combination with our flexible upcoming interpreter to gather type and other information at *runtime*. This dynamically gathered information can later be used to generate C, ObjC or other source code. Or for optimizations debugging and whatnot. So we wouldn't generate from a static Parse-Tree only but almost always in combination with the runtime information we gather during executing the tests. Thus we can use the tests not only for producing a robust code base but for generating code, specializations and optimizations of all kinds. I hope that with this approach we can avoid many "hints" which we might otherwise be inclined to hardcode. Better invest the time in good tests. For this to work we should have a scheme which makes it easy to get from a unittests to its correlating function/method/class/module and vice versa. Although not strictly neccessary i'd like to be able to navigate in both directions at runtime, also. and-how-do-we-test-the-tests-ly y'rs, holger

1 0

Memory management
by Richard Emslie 16 Feb '03

16 Feb '03

Hi all, This is my first post. I have been enjoying lurking away here, reading the various threads has been interesting despite having to occasionally peel my brain off the frying pan ;-) I have a naive question which I hope some kind sole will help me out on. I have been waiting for memory management (lower level than garbage collecting / ref counting) to be discussed. I'm probably barking up the wrong tree - but a "pypy python" object has to be resident in memory somehow - and this needs a physical representation and heap allocation / deallocation presumably? Perhaps this is a post-bootstrap issue as we can piggy back on the existing CPython infra-structure during development - but as far as I can reason this will have to be ultimately handled by a minimal statically compiled python which we do we do use in bootstrapping our pypy python? Is this true? Apologies if this has already been discussed - or I am plain misunderstanding how things will work. :-) Good luck with the sprint! Thanks, Richard

4 4

Re: [pypy-dev] repository access
by Laura Creighton 15 Feb '03

15 Feb '03

Windows users who would like to connect to subversion through internet explorer can try TortoiseSVN http://tortoisesvn.tigris.org/ I haven't used that myself, but I have seen its big brother TortoiseCVS in action http://www.tortoisecvs.org/ and it makes CVS one heck of a lot easier on Windows. Laura Creighton

1 0

Object model
by Armin Rigo 15 Feb '03

15 Feb '03

Hello everybody, I have been experimenting a bit about that distinction I tried to draw between "application-level" and "interpreter-level" objects. I still think the idea to be essential but I "reorganized" my thoughts a little bit. Now I believe in following the idea that CPython's main loop does not assume anything at all about the PyObjects it manipulates, but does every operation via calls to library functions like PyNumber_Add(). All it does is moving PyObjects around, mainly between the stack and the locals. Similarily, our own main loop should not assume anything about the objects it stores in the stack and the locals, not even that they have some well-known methods. Instead, we would provide it with the equivalent of PyNumber_Add() & co. Now the fun part of it is that there is no reason to use only one fixed set of functions to do the operations. There are several things we want to be able to do with our interpreter, and these will probably correspond to different ways of "seeing" what an object is. We might try several ways to implement the objects, for example trying different implementations for what the application will always see as a list. And even, we might want to have several concurrent "object spaces" existing at the same time, e.g. to implement the idea of multiple VMs in the same process. Thus it seems natural to define a class ObjectSpace, that defines methods like PyNumber_Add() and PySequence_GetItem(). Each frame would have an ''objectspace'' attribute that holds the space in which it runs. (Maybe shorter, more Pythonic names are better, e.g. add() and getitem().) Thus an object space is: * a way to implement objects; * from the interpreter's point of view, it is the "library" needed to do any operation; * from the application's point of view, it is essentially invisible. Methods: * add(x,y), getitem(x,y), etc. for the operations, that take two "objects" (black boxes for the interpreter) and returns an "object" (black box too). * type(x), taking a black box object and returning a black box object as well. * wrap(a), taking a *normal* object and putting it into a black box. Its what the interpreter does for a LOAD_CONST, for example: it has a real Python object and wants to send it to the object space. For example, if the object space has a custom implementation "class MyList" for lists, then wrap([1,2,3]) should create a MyList instance. * unwrap(x) is the inverse operation. Used by the interpreter in the rare cases where it really needs to observe the object. For example, for a conditional jump, after it obtained the truth-value with (say) a call to the truth() method, it must pull the result out of the object space to know whether it is really False or True. For example, the straightforward object space that just uses real Python objects to implement themselves would be defined like that: class BorrowingObjectSpace(ObjectSpace): def wrap(self, x): return x def unwrap(self, x): return x def add(self, x, y): return x+y def type(self, x): return type(x) ... Cool things can be imagined with other kinds of object spaces. We may even think about a distributed interpreter, running on one machine with object spaces that actually reside on other machines! With this perspective, wrap(x) sends x to the remote machine and returns a handle to it, and unwrap(x) downloads from the remote machine the object whose handle is x. We may even have several concurrently-running object spaces that could communicate, e.g. running a frame in a different object space and marshalling arguments and the return value! Looks like a cool point of view, doesn't it ? A bientôt, Armin.

4 6

Re: [pypy-dev] Scons?
by Bram Moolenaar 14 Feb '03

14 Feb '03

Holger Krekel wrote (to Charles Crain): > As we plan to run on top of CPython first it makes a lot > of sense to have SCons drive the CPython build and thus > get rid of make/configure dependencies. It's also a good > release goal for our sprint. The real challenge here is > to have a scheme for taking current CPython-cvs as an > "upstream" and have a system for keeping in > sync what we take out of CPython into our CPython-PyPy. As an alternative, you might want to look into A-A-P. Like SCons it is a Python based build system. The main difference is that SCons uses a file with Python syntax while A-A-P uses a syntax that looks more like a Makefile. In A-A-P you can use Python script just about everywhere, but the dependencies are specified like in a Makefile. One of the advantages is that you can specify a list of files without quotes or commas (you do need to use quotes when a file name contains a blank). Like SCons, A-A-P is in an alpha stage. SCons has more support for compilers and languages. A-A-P has support for uploading, downloading and CVS. I'm currently working on a GUI IDE for A-A-P. Most info can be found on the web site: http://www.A-A-P.org/ Have a look at the examples: http://www.A-A-P.org/examples.html Ask me if you have further questions. -- hundred-and-one symptoms of being an internet addict: 225. You sign up for free subscriptions for all the computer magazines /// Bram Moolenaar -- Bram(a)Moolenaar.net -- http://www.Moolenaar.net \\\ /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html ///

2 2

Re: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON?
by Laura Creighton 14 Feb '03

14 Feb '03

In a message of Thu, 13 Feb 2003 21:32:30 +0100, holger krekel writes: >[Laura Creighton Wed, Feb 12, 2003 at 08:47:09PM +0100] >> This is precisely the sort of thing that Nat Tarkington said >> he wanted for the Python track -- new community efforts, new >> implementations. >> >> Who is going to OSCON here besides me? > >I might go but it's impossible to say now. It's quite >an expensive and faraway-from-here trip. > >Anyway, I am fine with you doing a proposal. Have the >people actually doing the presentation to be on the proposal? >If so then put me on although it's less than 50% that >i will make it. And remember: flexibility and simplicity >first, then performance :-) > > holger No, anybody can be on the proposal. We can put the entire membership of pypy-dev on, for instance. The person who makes the proposal doesn't have to be the person or persons who make the paper, or even that does the presenting. The only thing is the deadline for the proposal is Feb 15th, and what I need is for it to be ok with people here that if I make one, and if it gets accepted, that some paper will happen which Jacob or I could present. Other people are, of course, welcome to do the presenting, if they are attending. Laura

1 0