Mailman 3 April 2004 - pypy-dev

Brainstorming at Python UK
by Armin Rigo May 6, 2004

May 6, 2004

Hi, Here is as promized a few words about the "brainstorming" at Python UK about the annotation stuff. The general idea is to leverage what we have already built for translation, including the annotation-propagation framework, but replace the annotations themselves with something more reasonable. In other words if you don't know exactly what annotations are, fine: now we want to drop them. "Annotations" were the mean through which typing information about specific variables were stored. We'd like to replace them with explicit "abstract objects", which would be instances of simple classes like: class SomeObject: "Nothing is known about this value." class SomeInteger: def __init__(self, start=-sys.maxint-1, end=sys.maxint+1): self.start = start self.end = end class SomeList: def __init__(self, anyitem): self.anyitem = anyitem # another SomeXxx instance So the "type inference" phase of the translation would take a control flow graph as input, as it currently does; then its goal is to associate one SomeXxx instance to each variable. This information can then be used by the code generator (Pyrex or Lisp). A new idea inspired by Starkiller, which is an important difference with the old way annotations worked, is that all the SomeXxx instances are always immutables. For example, SomeInteger(0,10) is and will always mean an integer in range(0,10). If later a more general value is found that could arrive into the same variable, then the variable gets associated to a new, more general instance like SomeInteger(-10,10). This change triggers a recomputation of the type inference, and the new SomeInteger(-10,10) will eventually propagate forward to wherever the variable is used. Here is an example: Block(v1,v2,v3): v4 = v1+v2 v5 = v4+v3 jump to Block2 with arg (v5) If the first time Block is entered, v1, v2 and v3 all contained SomeInteger(0,10), then type inference will deduce that v4 contains SomeInteger(0,19) and v5 contains SomeInteger(0,28), and this SomeInteger(0,28) will be sent to Block2 as its input argument, and Block2 will be analysed accordingly. But if later, we find out that Block can also be entered with v1 being SomeInteger(-5,5), then we compute the union of this new value with the previous one: SomeInteger(0,10)|SomeInteger(-5,5) = SomeInteger(-5,10). Then we re-propagate the values into Block, disregarding what we previously found, and get SomeInteger(-5,28) into v5. Then we send this value to Block2, and the same process is repeated there: if this is more general than what we have already seen for Block2, then we generalize and recompute Block2. This process continues until we reach a fixpoint. (Of course in practice I suppose that we won't have things like SomeInteger(0,10), but rather e.g. SomeInteger(0,sys.maxint+1) to mean a non-negative integer. We could just define SomeInteger to have a flag "can I be negative". The purpose of knowing when integers are not negative is typically to simplify list indexing: lst[i] is easier to implement if we know that i>=0.) Having immutable SomeXxx instances even for mutable objects is very useful for the object-oriented part of the analysis. Say that a SomeInstance would represent a Python instance, in the old-style way: a __class__ and some attributes. The SomeInstance is created at the point in the flow graph that creates the instance: v2 = simple_call(<class X>) This would register somewhere that the class X can be instanciated from this point in the flow graph. It would then create and store into v2 a SomeInstance of class X, which initially has no known attribute. If an attribute is added later on v2, then the SomeInstance detects it is not general enough. It kills the inference process; it records the existence of the new attribute in some data structure on the class X; and it marks all the creation point of instances of X as invalid again. This will restart the type inference from there, and the creation points will now build SomeInstances that can accomodate the extra attribute. The same occurs if we figure out that an existing attribute wasn't general enough, e.g. when we store a possibly negative value into an attribute known to be SomeInteger(0,10). It seems that we will end up cancelling and restarting large parts of the analysis over and over again this way, but it may not be a big problem in practice: we can expect that a lot of information about the attributes will be known already after the __init__ call completed. We may stop and re-analyse __init__ itself 7 times consecutively if it defines 7 attributes on self, each time progressing a bit further until the next new attribute kills us, but that shouldn't be a big problem because it is fairly local (but we'll have to try to analyse bigger programs to really know). This is much cleaner this way than with the annotation stuff. A bientôt, Armin.

2 4

Slides: how Psyco works
by Armin Rigo April 29, 2004

April 29, 2004

Hello, I have put on-line some slides from the Python UK conference at ACCU 2004, explaining how Psyco works. It is actually a Pygame application... As far as I can tell, it is the first time someone in the room actually understood something at the end :-) http://psyco.sourceforge.net/ I hope it should help to make Psyco a bit less mysterious, and also explain why it is difficult to have a general idea about what kind of speed-up you can expect for specific kinds of code: it is, after all, a pretty low-level "local" process that Psyco does, and it sometimes pays off and sometimes not. It also explains why this process is very much like the usual interpretation that CPython does. If you think about it you might see how useful it would be for Psyco to build on top of an interpreter in a better language than C (i.e. one that can be analysed, not just blindly run). A bientôt, Armin.

3 2

talk needed at europython
by Michael Hudson April 27, 2004

April 27, 2004

There's a general expectation that there's going to be a talk on PyPy at this year's EuroPython, so I think we should give (at least) one. Provisional idea: title "PyPy: What it is, where it's going and how we're going to pay for it", a 60 minute talk on the language track in three 20 minute sections, possibly with a different speaker for each section. Thoughts? If someone wants to give a more detailed talk on the whole funding thing, that would be cool, but it probably wouldn't be on the lanugage track... Cheers, mwh -- I think perhaps we should have electoral collages and construct our representatives entirely of little bits of cloth and papier mache. -- Owen Dunn, ucam.chat, from his review of the year

6 9

Re: [pypy-dev] talk needed at europython
by Michael Hudson April 26, 2004

April 26, 2004

Armin Rigo <arigo(a)tunes.org> writes: > Hello, > > On Sat, Apr 24, 2004 at 11:13:20PM +0200, Anna Ravenscroft wrote: >> > I'm rather shocked that this hadn't been planned earlier. :) >> >> Um, maybe they were too busy planning the sprint and trying to *get* the >> funding... ;-) > > We seem to be in a "catch-up-with-real-life" mood after this EU thing... For > the sprint, I am more or less assuming that the dates are as specified on the > Wiki page and people should use the Wiki page to say if they want > accomodation. Registration on the conference should then be done without > asking for accomodation; we'll sort it out globally for everybody interested > in sprinting. Am I correct? I'm not sure. I've registered for the conference and booked accomodation for the maximum time you can on the conference website. > A PyPy track on EuroPython would be needed indeed. If no other comment comes > up I'll officially submit one for Michael's language track. Please do. Details can be filled in/changed outright later on. Cheers, mwh -- I don't remember any dirty green trousers. -- Ian Jackson, ucam.chat

5 5

Re: [pypy-dev] python/pypy hacking before EuroPython
by Laura Creighton April 20, 2004

April 20, 2004

In a message of Thu, 08 Apr 2004 16:04:33 +0200, holger krekel writes: >[Laura Creighton Thu, Apr 08, 2004 at 03:54:15PM +0200] >> In a message of Thu, 08 Apr 2004 15:34:21 +0200, holger krekel writes: >> >But as a remedy the idea now is to do a 3+7 day coding sprint >> >before EuroPython in Goetheburg/Sweden: >> > >> > 28th of May till 30th of May: Free Python hacking >> > 31st of May till 6th of June: PyPy sprint >> > 7th of June till 9th of June: EuroPython! >> > >> >Be assured that it will be a wild ride that you don't want >> >to miss :-) I guess our main focus will be on translation to >> >low level code to finally get a working C prototype of PyPy. >> >There are however, many diverse areas of interest and we will >> >give a thorough introduction and help to newcomers. >> >> I'd sort of prefer the 3 free days to be before EP, and the 7 days to >> start the 28th. That way I can go nuts about the conference the 3 days >> before not in the middle of pypy. do you have a reason why you want it >> this way? > >Well, i figured i wouldn't like to force everybody interested in >a PyPy sprint into another three day free hacking thing ... >actually i wasn't sure who apart from Armin and me are crazy >enough and have enough time to do the extra 3 days :-) >It's surely open to discussion though. > >cheers, > > holger Well, last EP, it was only at EP that I found the people I wanted to hack with, so having 3 free days AFTER ep might be best. Since we have all the sprint rooms booked with plan of before and after sprints, this be easier in some ways as well. Laura

5 4

source code
by Thomas Heller April 15, 2004

April 15, 2004

I tried to loo at the pypy source code, but neither the viewcvs link http://codespeak.net/viewcvs.cgi/pypy/trunk/src/pypy worked, nor could I check out with svn: c:\>svn --version svn, version 0.24.2 (r6284) compiled Jun 19 2003, 05:44:59 Copyright (C) 2000-2003 CollabNet. Subversion is open source software, see http://subversion.tigris.org/ The following repository access (RA) modules are available: * ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol. - handles 'http' schema - handles 'https' schema * ra_local : Module for accessing a repository on local disk. - handles 'file' schema * ra_svn : Module for accessing a repository using the svn network protocol. - handles 'svn' schema c:\>svn co http://codespeak.net/svn/pypy/trunk/src/pypy A pypy\interpreter svn: RA layer request failed svn: REPORT request failed on '/svn/pypy/trunk/src/pypy' svn: REPORT of '/svn/pypy/trunk/src/pypy': 409 Conflict (http://codespeak.net) c:\> What's the problem? Thanks, Thomas

3 4

PyPy talk(s) at EuroPython
by Armin Rigo April 15, 2004

April 15, 2004

Hello, There are two potential PyPy talks that would be nice to have at EuroPython. The technical one: maybe a 30-minutes talk could quickly survey what we already had some time ago (already talked about e.g. in last year's EuroPython), and then show the new stuff like the control flow analysis, and finally tell what cool things we have done in the sprints just before. There is also a Social skills tracks: """This track is not directly related to Python but definitely addresses a problem common to all communities of software developers, a.k.a. geeks: that of having to communicate with other groups/individuals who are driven by totally different priorities, such as corporate/academic politics, finances, etc. """ We should probably explain how to get EU funding there. A bientôt, Armin.

1 0

Re: [pypy-dev] i just had the book FLOWS reccomeenmded to me
by Laura Creighton April 15, 2004

April 15, 2004

In a message of Wed, 14 Apr 2004 11:26:53 +0200, Martijn Faassen writes: <lots of neat stuff I agree with> >I do agree with Alex's thesis that fairly uncreative activitiy is a >necessary ingredient of human society in many cases. I'm glad we don't >all want to be programmers working on a new software system; this way >some of us can be the grunt programmers, the taxi drivers and the cooks >of this world. > >Regards, > >Martijn If I am correct in inferring from this that you think of cooking as one of the 'uncreative activities' ..... shame shame shame. I must cook for you some day. :-) Laura

2 1

Re: [pypy-dev] i just had the book FLOWS reccomeenmded to me
by Laura Creighton April 14, 2004

April 14, 2004

In a message of Tue, 13 Apr 2004 13:39:44 +0200, Alex Martelli writes: >On Saturday 10 April 2004 12:05 am, Edward K. Ream wrote: >> > Why do uncreative people live at all? >> >> Why ask why? It's their problem :-) > >I disagree: we, the uncreative, provide most of the indispensable support > and >follow-through for most creative people's "sparks". <snip> somewhere along the line, we have the interesting phenomenon that Alex Martelli doesn't think of himself as creative ... meanwhile, he goes about life creating a lot. :-) You must think that creativity is something different than what happens to you when you are creating things. How very odd. I would say that all of those support people are creators too. However, the joy of 'helping' might be more of what motivates them than the joy of 'creating'. I am curious as to what happens to those people who are getting satisfaction out of their lives while not being creative, because I understand them so poorly This will make it hard for us to build attractive work environments for them. Whatever it is that they want, we probably do not have. Laura

3 2

Re: [pypy-dev] i just had the book FLOWS reccomeenmded to me
by Edward K. Ream April 13, 2004

April 13, 2004

> > Why ask why? It's their problem :-) > > I disagree: we, the uncreative, provide most of the indispensable support > and follow-through for most creative people's "sparks". Interesting reply. I doubt that we are in great disagreement here. My question "why ask why" was meant to convey that we needn't get too concerned about what other people do: there is little we can do about it anyway. Another reason I'm not too concerned about the question is that I think most people are creative in ways that may not be apparent to the casual observer. Perhaps we have slightly different views of the "scale" of creativity. In my view, there is potential for creativity almost everywhere, including in the "long, careful, systematic, painstaking, often-boring and always-tiring work, implied by that 99% of perspiration." In my own work, the "big aha" happened 9 years ago, and it took about 10 minutes of using a prototype (The MORE outliner) to see that a programming style based on outlines would work. That programming style has remained almost unchanged ever since. Was that the end of creativity? I don't think so! There is lots of room for creativity in "dotting the i's". In fact, I think this is where almost all creativity is. It's not particularly glamorous, and it _is_ real creativity. Again, I doubt we are in much disagreement. Alex, surely your life must be highly creative, even if most of it seems like "perspiration" :-) I wasn't trying to dismiss people who aren't creative; I was trying to dismiss worrying about whether people are creative or not. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo(a)charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html --------------------------------------------------------------------

2 1