Mailman 3 June 2005 - pypy-dev

Re: [pypy-svn] r13606 - pypy/dist/pypy/translator
by Christian Tismer 19 Jun '05

19 Jun '05

arigo(a)codespeak.net wrote: > Author: arigo > Date: Sun Jun 19 19:07:32 2005 > New Revision: 13606 > > Modified: > pypy/dist/pypy/translator/geninterplevel.py > Log: > Use the SSI_to_SSA optimization in geninterplevel. This allows the generated > functions to look nicer, with less copying around all over the place the same > value. Nice change! This is also a step towards scopedgraphs, which will probably be yet another entry in backendoptimization.py The main difference would be to identify subgraphs with a single entry and exit point. These can then be rendered by nesting them into a block and avoiding to transfer the implied variables over the entry link at all. ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

Re: [pypy-svn] r13583 - in pypy/dist/pypy/translator/llvm2: . test
by Armin Rigo 19 Jun '05

19 Jun '05

Hi, On Sat, Jun 18, 2005 at 06:28:30PM +0200, cfbolz(a)codespeak.net wrote: > - added flow graph transform remove_same_as, because same_as is > incompatible with SSA in LLVM. I'm not sure I see why 'same_as' is incompatible with LLVM? It's not an "assignment" operation, it's just an operation that creates a new result variable just like all the other ones. The optimization is nice, though. A bientot, Armin

3 2

Status update
by Armin Rigo 15 Jun '05

15 Jun '05

Hi, An update on the translator status. Dark obscure stuff has been mostly sorted out. Exceptions work all the way from the annotator to the rtyper to GenC. There is the beginning of a low-level interpreter now. Still missing are: * GenC knows little about float operations. Need to complete translator/c/float_include.h. * Almost all the topics from the rtyper, from the previous status report: - (hard) prebuilt constants -- PBCs -- and dicts with PBC keys - (medium) dicts with string keys, and iterators over them - (easy) iterators over strings - (medium) iterators over tuples - (varying) all built-in functions listed in annotation/builtin.py - (mostly easy) a few more operations: id, issubtype, str(?), is - (DONE) add_ovf --> int_add_ovf - (easy) list methods reverse, pop, insert, index(?) - (easy) tuple concatenation - (easy) string methods startswith(1-arg), endswith(1-arg), join (possibly only on the empty string ''), string slices - (easy) equality and non-equality are generally missing - (medium) limited form of string formatting: 'constant template' only which should only generate a sequence of simple operations like concatenation and str() on integers * The lltype model is, hum, mostly stable but still has changed bits and pieces all the time. One important change already done is that arrays can be of primitive types directly. This means that rlist.py and rstr.py can be simplified a bit: for example, in rstr, STR can be defined with 'chars' as an Array(Char). It makes reads and writes to characters more natural to write in the ll_() functions. Another simplification is the operation cast_pointer, which replaces cast_parent, and which can cast freely pointer types as long as the cast is done (in any direction) between a structure and a substructure which is the first field of the structure -- or the 1st field of the 1st field, etc. This made rclass.py simpler. * A missing piece of lltype that we'll work on soon is proper support for deallocation. The problem right now is that if you have a pointer to a GcStruct 'B' which is actually the 1st field of a larger GcStruct 'A', and this pointer goes away, GenC generates a decref; if the refcount reaches zero, it frees the structure beliving that is really a 'B' structure. This means that the extra part of 'A' is ignored and just leaks away. To solve this, we need to introduce minimal support for run-time types -- just some kind of opaque pointer that describes what the real, "large" GcStruct is. * lltype will also need to grow a new primitive type, UnicodeChar. A bientot, Armin

1 0

A small patch
by Ben.Young＠risk.sungard.com 15 Jun '05

15 Jun '05

Hi Everyone, After all my pointless question asking, I thought it would be best if I actually contributed something. Would someone mind applying this patch what allows more tests to compile on cygwin, using VC++ as the compiler. It seems to object to empty array initialisers. Cheers, Ben Index: translator/c/node.py =================================================================== --- translator/c/node.py (revision 13434) +++ translator/c/node.py (working copy) @@ -279,6 +279,9 @@ if self.T.OF == Void: yield '\t%d' % len(self.obj.items) yield '}' + elif len(self.obj.items) == 0: + yield '\t0, NULL' + yield '}' else: yield '\t%d, {' % len(self.obj.items) for j in range(len(self.obj.items)):

2 2

Re: [pypy-svn] r13371 - pypy/branch/pycompiler/bin
by Armin Rigo 14 Jun '05

14 Jun '05

Hi Adrien, On Tue, Jun 14, 2005 at 11:12:15AM +0200, adim(a)codespeak.net wrote: > Log: > added readline / history / tab support for translator shell > (fed up with always retyping the same things ... ) Do you mind if this is copied to the trunk? I guess it would be generally useful. The history file could be stored locally or in the _cache directory. Armin

2 2

Re: [pypy-svn] r13328 - pypy/dist/pypy/translator/test
by Sanghyeon Seo 14 Jun '05

14 Jun '05

On 6/13/05, arigo(a)codespeak.net <arigo(a)codespeak.net> wrote: > +py.test.skip("the Translator and back-ends depend on too many conditions " > + "to test the back-ends generically") > +# e.g. some back-ends require annotation and/or specialization to be done > +# while some require it *not* to be done. GenCL for example has no > +# chance to understand a specialized graph. (That's why the test still > +# used to pass: forty_two is basically an empty function, so > +# specializing it makes no difference. Now the RTyper produces a few > +# helper functions in all cases.) I am fine with getting rid of current GenCL if it blocks any of PyPy's development. Pity I don't have enough free time for PyPy these days. Seo Sanghyeon

3 4

planning the 0.6.2 PyPy release
by Ben.Young＠risk.sungard.com 14 Jun '05

14 Jun '05

Hi Armin Just a quick followup question.... >> In a week or so I may have a little time to do a bit of coding. Is there >> anything specific I should look at? For instance I noticed that at the >> moment the list rep resizes exactly. The python version (and stuff like >> std::vector in c++) have a overallocation stratagy thay minimises >> allocation calls. Would this be an idea I could try to implement in >> rpython? >An excellent question! It's slightly strange, given that >pypy.objspace.std.listobject already does over-allocation, that we might >need it again at this lower level. It's possible that it would make >sense for some of the lists that we have around, and not for others -- >the hard question will eventually be to decide when to use it or not. >But yes, I guess that before deciding that, we'd need to have both >options written :-) (There are still some open questions there; for >example, it's not too difficult to detect list comprehensions by looking >at the flow graphs; then instead of heuristic over-allocation for the >list that is being built, it's often possible to determine in advance >the size of the list -- e.g. because it will be the same length as >another list that we iterate over.) Does this mean that when you eventually come to *dynamically* annotate code that lists will be an annotated version of the std.listobject, or do you expect it to use the rpython typer as well? Thanks for answering all these questions, it definitely is making things clearer in my mind. Cheers, Ben

3 5

postponing 0.6.2/the move to CPython 2.4.1
by holger krekel 13 Jun '05

13 Jun '05

Hi folks, well, it seems our early 0.6.2 planning this month may have been a bit too optimistic. Armin, Samuele, Christian and me discussed today that we should not go for the move to CPython 2.4.1 before EuroPython. The main reasoning is that it requires an unknown amount of effort and might also divert from the translation focus because of little issues popping up everywhere. Instead, the idea is to experiment with the 2.4.1-move at the Post-EuroPython sprint. After that we should have a better picture of when it makes sense to go for a new release. Basically we should stably work with the 2.4.1 baseline for some time before doing a release. However, we think that improving the documentation before EuroPython and the sprint still makes sense. But that doesn't require a new release per se. So unless there is serious disagreement, the 0.6.2 release and the 2.4.1 move will not be tackled in June. At least we discovered the problem before being well into the release process so that could be called progress :-) cheers, holger

1 0

Fw: [pypy-svn] r13221 - in pypy/dist/pypy/rpython: . test
by Ben.Young＠risk.sungard.com 13 Jun '05

13 Jun '05

(sorry if this appears twice, used wrong reply-to header) Hi Arigo, Looking at these commits, I just have a quick question. Can the ll_ functions call each other? If they could then all the ll_delitem functions could just delagate to the noneg versions. Also, does the bounds checking happen at this level, or at a lower one. In a week or so I may have a little time to do a bit of coding. Is there anything specific I should look at? For instance I noticed that at the moment the list rep resizes exactly. The python version (and stuff like std::vector in c++) have a overallocation stratagy thay minimises allocation calls. Would this be an idea I could try to implement in rpython? Cheers, Ben pypy-svn-bounces(a)codespeak.net wrote on 09/06/2005 09:53:32: > Author: arigo > Date: Thu Jun 9 10:53:30 2005 > New Revision: 13221 > > Modified: > pypy/dist/pypy/rpython/rlist.py > pypy/dist/pypy/rpython/test/test_rlist.py > Log: > list.delitem; tests. > > > Modified: pypy/dist/pypy/rpython/rlist.py > ============================================================================== > --- pypy/dist/pypy/rpython/rlist.py (original) > +++ pypy/dist/pypy/rpython/rlist.py Thu Jun 9 10:53:30 2005 > @@ -88,6 +88,14 @@ > llfn = ll_setitem > return hop.gendirectcall(llfn, v_lst, v_index, v_item) > > + def rtype_delitem((r_lst, r_int), hop): > + v_lst, v_index = hop.inputargs(r_lst, Signed) > + if hop.args_s[1].nonneg: > + llfn = ll_delitem_nonneg > + else: > + llfn = ll_delitem > + return hop.gendirectcall(llfn, v_lst, v_index) > + > class __extend__(pairtype(ListRepr, SliceRepr)): > > def rtype_getitem((r_lst, r_slic), hop): > @@ -143,13 +151,39 @@ > i += len(l.items) > return l.items[i].item > > +def ll_setitem_nonneg(l, i, newitem): > + l.items[i].item = newitem > + > def ll_setitem(l, i, newitem): > if i<0: > i += len(l.items) > l.items[i].item = newitem > > -def ll_setitem_nonneg(l, i, newitem): > - l.items[i].item = newitem > +def ll_delitem(l, i): > + if i < 0: > + i += len(l.items) > + newlength = len(l.items) - 1 > + newitems = malloc(typeOf(l).TO.items.TO, newlength) > + j = 0 > + while j < i: > + newitems[j].item = l.items[j].item > + j += 1 > + while j < newlength: > + newitems[j].item = l.items[j+1].item > + j += 1 > + l.items = newitems > + > +def ll_delitem_nonneg(l, i): > + newlength = len(l.items) - 1 > + newitems = malloc(typeOf(l).TO.items.TO, newlength) > + j = 0 > + while j < i: > + newitems[j].item = l.items[j].item > + j += 1 > + while j < newlength: > + newitems[j].item = l.items[j+1].item > + j += 1 > + l.items = newitems > > def ll_concat(l1, l2): > len1 = len(l1.items) > > Modified: pypy/dist/pypy/rpython/test/test_rlist.py > ============================================================================== > --- pypy/dist/pypy/rpython/test/test_rlist.py (original) > +++ pypy/dist/pypy/rpython/test/test_rlist.py Thu Jun 9 10:53:30 2005 > @@ -30,6 +30,21 @@ > assert ll_len(l) == 4 > check_list(l, [42, 43, 44, 45]) > > +def test_rlist_set_del(): > + l = sample_list() > + ll_setitem(l, -1, 99) > + check_list(l, [42, 43, 44, 99]) > + ll_setitem_nonneg(l, 1, 77) > + check_list(l, [42, 77, 44, 99]) > + ll_delitem_nonneg(l, 0) > + check_list(l, [77, 44, 99]) > + ll_delitem(l, -2) > + check_list(l, [77, 99]) > + ll_delitem(l, 1) > + check_list(l, [77]) > + ll_delitem(l, 0) > + check_list(l, []) > + > def test_rlist_extend_concat(): > l = sample_list() > ll_extend(l, l) > @@ -110,4 +125,13 @@ > def dummyfn(): > l = [5, 6, 7, 8, 9] > return l[:2], l[1:4], l[3:] > - rtype(dummyfn).view() > + rtype(dummyfn) > + > +def test_set_del_item(): > + def dummyfn(): > + l = [5, 6, 7] > + l[1] = 55 > + l[-1] = 66 > + del l[0] > + del l[-1] > + rtype(dummyfn) > _______________________________________________ > pypy-svn mailing list > pypy-svn(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn >

2 2

Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005
by Armin Rigo 12 Jun '05

12 Jun '05

Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005 ====================================================== The next PyPy sprint is scheduled right after EuroPython 2005 in Gothenborg, Sweden. Its main focus is translation to lower level backends but there are also other possible topics. We'll give newcomer-friendly introductions. To learn more about the new PyPy Python-in-Python implementation look here: http://codespeak.net/pypy On a side note, there are a number of sub projects that may be interesting for participating in google's summer-of-code event (deadline June 14th!). The PyPy group is willing to mentor projects that have some link with PyPy, so if you are accepted in such a project, the sprint could also serve as a good meeting and kick-off point. Further down you'll find some examples, but there are certainly more and bigger ones :-) Goals and topics of the sprint ------------------------------ The main, though not the only, focus of the sprint will be on the "translation" aspect of PyPy. The goal here is to progress towards a completely translated PyPy. How much will already have been done before EuroPython is unknown; as a guess, we will be left with: - completing the "rtyper", the piece of code that assigns low-level C-like types to high-level RPython objects (lists, dicts, instances, etc.) and low-level control flow graphs to high-level ones; - polish off the GenC and GenLLVM back-ends, responsible for turning the low-level C-like flow graphs into real C or LLVM source code. See http://codespeak.net/pipermail/pypy-dev/2005q2/002136.html for more information (10th of June status). Non-translation-related topics are welcome too. Here are some suggestions from the issue tracker (https://codespeak.net/issue/pypy-dev/): - integrate the parser module, possibly making it RPython conformant; - rewrite in Python a C module you are familiar with (partial list of missing/incomplete modules: os, math, array, regular expressions, binascii...) - implement Python 2.3's import hook extensions (zip-imports etc.) - fix Windows-related issues, '%'-formatting rounding errors, add missing docstrings on app-level built-in types and functions, etc. - weakrefs (but this requires discussion and planning on pypy-dev before the sprint! feel free to start such a discussion, though.) Location & Accomodation ------------------------ The sprint will be held in the former Math Center building near the crossing of Gibraltargatan and Eklandagatan. Entrance is on the middle of the side facing Gibraltargatan. The doors to the building are normally locked, so you need the phone number of somebody inside to get in. Instructions on whom to call will be posted on the door. The sprint will be co-located with several other sprints. See the `EuroPython Wiki`_, to find out what other sprints will be running. Nearest, and probably cheapest is to book accomodation at SGS Veckobostäder through the Europython website. This option will be available until about 20 June. .. _`EuroPython special accomodation`: http://www.europython.org/sections/accomodation/special_accomodation .. _`EuroPython Wiki`: http://www.europython.org/sections/sprints_and_wiki Exact times ----------- The public Pypy sprint is held Friday 1st July - Thursday 7 July 2005. Hours will be from 09:00 until people have had enough. It's a good idea to arrive a day before the sprint starts. (There is a sprint for people who are familiar with the Pypy codebase before Europython as well. This will be held at Jacob & Laura's home on Götabergsgatan 22.) Network, Food, currency ------------------------ Sweden is not part of the Euro zone. One SEK (krona in singular, kronor in plural) is roughly 1/10th of a Euro (9.15 SEK to 1 Euro). There are some pizzerias, kebab places and the like close to the venue. Their food is edible and cheap, but not very good. For good food, you need to go downtown. You need a wireless network card to access the network. You will be issued a login to the Chalmers NOMAD network. This will allow you to use access points all over Chalmers. However, we can likely provide a wireless/ethernet bridge. Sweden uses the same kind of plugs as Germany. 230V AC. Registration etc.pp. -------------------- Please subscribe to the `PyPy sprint mailing list`_, introduce yourself and post a note that you want to come. Feel free to ask any questions there! .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint -- Armin Rigo & the PyPy team

1 0