Mailman 3 June 2004 - pypy-dev

Re: [pypy-svn] r5369 - in pypy/trunk/src/pypy: scripttest scripttest/test tool
by holger krekel July 14, 2004

July 14, 2004

Hi Rocco, [rocco(a)codespeak.net Sun, Jun 27, 2004 at 10:53:28PM +0200] > Author: rocco > Date: Sun Jun 27 22:53:28 2004 > New Revision: 5369 > > Added: > pypy/trunk/src/pypy/scripttest/ (props changed) > ... > Log: > * tool/app_level_diff.py > > I hacked together a little tool for running a script at app level, and checking > its output against an expected output file. (Sort of like regrtest.py) I added > a feature where the files can be sectioned off into multiple parts, to reduce > disk clutter. > > * scripttest directory I guess this at least should not live directly in src/pypy but in some subdirectory (e.g. tool) or at 'src/scripttest' or something similar. And btw, it's preferable if you talk to some other developer or best to pypy-dev before adding a whole bunch of things :-) > The start of using app_level_diff as a testing tool - I've converted the > CPython regression test test_grammar to it's use. Sectioning is incomplete - > the parts that are sectioned are mainly to isolate failures > > I've also started to do a script made from the examples given in Guido's > tutorial. Incomplete as of now. Can you give some examples of usage? I am a bit of a loss what exactly you are trying to do here. cheers, holger

2 3

[FW: Re: [pypy-dev] Patches?]
by Seo Sanghyeon July 2, 2004

July 2, 2004

Oops, replied in private by mistake... ----- Forwarded message from tinuviel ----- To: Ben Young <ben(a)transversal.com> Subject: Re: [pypy-dev] Patches? On Wed, Jun 30, 2004 at 11:08:38AM +0100, Ben Young wrote: > Hi, > > I have been following the pypy-dev mailing list for some time, as well as > maintaining a anon svn checkout of the pypy source and am really excited by > this project (in fact it has inspired me to write my own experimental > language in python along similar lines!). Welcome to PyPy! > I was just wondering whether the > project is considered stable enough to accept patches and/or bug reports from > people who are not directly developers (i.e me!). I have been playing with > getting pypy to run some of my own python programs, and although it seems to > get pretty close there are a couple of issues. PyPy is not exactly stable, but I think patches are welcome any time. > The first is in appspace/operator.py where if you import this in the trivial > object space then is gives errors about modifying a dictionary while > iterating over it. This can be fixed by the following patch > (patch follows) Fixed in r5407, but in a different way. In case you don't read the commit mailing list, here is the detail: http://codespeak.net/pipermail/pypy-svn/2004-June/002572.html In short, "#ugh" part is removed altogether. It was a hack, and it was there to make built-in function never-bound as in CPython. (See the checkin message.) If you don't understand what that means, try this: ==== from operator import add def add2(x,y): return x+y class X: pass X.add = add X.add2 = add2 print X().add(2,2) print X().add2(2,2) ==== CPython doesn't raise exception for X().add, but do for X().add2. That's because built-in function operator.add is never-bound. PyPy before commit does the same. PyPy after commit raise exception for both. > The second is in pprint. If you do > > >>>> import pprint > >>>> pprint.pprint([]) > > you get a massive traceback which I haven't yet quite worked out a fix for. This seems to work in the standard object space though. Regards, ----- End forwarded message -----

3 2

utest conversion tool question
by Laura Creighton July 1, 2004

July 1, 2004

I am still dizzy and sick, but I am trying to get something done anyhow. I have this question. If you start with self.assertEquals(f(x) + g(x) == q(x), 'Message to print when the assertion fails') you would like things to come back as assert f(x) +\ g(x) == q(x), 'Message to print when the assertion fails' It is the last, optional message which is the sticky part, otherwise you could say, just keep the original parentheses. But this does not work, as the following demonstrates: >>> assert 0 + \ ... 1 == 7, 'no it doesnt' Traceback (most recent call last): File "<stdin>", line 1, in ? AssertionError: no it doesnt >>> assert (0 + ... 1 == 7, 'no it doesnt') >>> The problem is that once you get into 'backslash pasting' mode, it is hard to know when you should stop. I was working on the assumption that if we have something that works like an expression, but doesn't work when you peal off the parens, i.e. it is multi-line, then you should paste the parens in. And this is mostly correct. But there is this problem. >>> s=''' ... aaa ... bbb ... ccc ... ''' >>> s '\naaa\nbbb\nccc\n' >>> >>> assert ''' ... aaa ... bbb ... ccc ... ''' != s Traceback (most recent call last): File "<stdin>", line 1, in ? AssertionError >>> thus if I get some code: self.assertNotEquals(''' aaa bbb ccc ''', s) I should not be pasting in the backslashes. But >>> parser.expr(s) Traceback (most recent call last): File "<stdin>", line 1, in ? File "<string>", line 3 bbb ^ SyntaxError: invalid syntax >>> .... suggestions on how to recognise triple-quoted strings? Apologies if I miss something obvious, my brain isn't really good for much yet. Laura

5 8

Patches?
by Ben Young June 30, 2004

June 30, 2004

Hi, I have been following the pypy-dev mailing list for some time, as well as maintaining a anon svn checkout of the pypy source and am really excited by this project (in fact it has inspired me to write my own experimental language in python along similar lines!). I was just wondering whether the project is considered stable enough to accept patches and/or bug reports from people who are not directly developers (i.e me!). I have been playing with getting pypy to run some of my own python programs, and although it seems to get pretty close there are a couple of issues. The first is in appspace/operator.py where if you import this in the trivial object space then is gives errors about modifying a dictionary while iterating over it. This can be fixed by the following patch Index: appspace/operator.py =================================================================== --- appspace/operator.py (revision 5402) +++ appspace/operator.py (working copy) @@ -164,7 +164,7 @@ # ugh: import types as _types -for n in globals(): +for n in list(globals()): v = globals()[n] if isinstance(v, _types.FunctionType): globals()[n] = staticmethod(v) The second is in pprint. If you do >>>> import pprint >>>> pprint.pprint([]) you get a massive traceback which I haven't yet quite worked out a fix for. Thanks for your help! Ben Young

1 0

Summary
by Armin Rigo June 29, 2004

June 29, 2004

Hello ! Here is at least a quick overview of the sprint changes... * Holger already talked about the big change, which was to use CPython's descriptor model as the object model. Now all classes, built-in or not, have a bunch of __xxx__ methods and properties, and the operations themselves (e.g. space.add()) are no longer multimethods but calls to these special methods, using CPython's rules, which you can see in objspace/descroperation.py. For example, space.add(w1,w2) works by looking up __add__ on the type of w1 and then __radd__ on the type of w2. This means that there isn't a lot of difference any more between built-in and user-defined classes, which is good. Note that multimethods are still used to populate the built-in classes, e.g. int.__add__() is created as a function object that dispatches to one of the add__Int_Xxx() functions. So basically every complex meta-class magic works just fine in PyPy now :-) Credits in alphabetical order: Holger, me, Michael, Samuele. * Samuele and Alex (and Holger on tests) implemented the package import system. * On the translator side, Anders and myself worked on code generation: it can produce things like classes with methods on them. Then Anders cleaned it up a lot and refactored the corresponding files and tests. Holger helped here too. * Anna worked on documentation. * We considered the unittest-to-utest switch a couple of time. utest seems ready now. At the end of the second week Laura and Jacob started working on an automatic translation tool for all our test_*.py files. Laura seems to have almost finished it by now. * Michael and Holger have put an (optional) HTTP server in PyPy, to handle these long tracebacks in a browser. I think it is still waiting for improvement. * A lot of small things went fixed all along the sprint, notaby by trying to run PyPy on top of itself. * Holger moved 'class complex' into __builtin__module.py. * Alex, Holger, Michael: String formatting ( % ) is done by PyPy now; I think it's not entierely complete yet. * Laura worked a bit more on her tool to figure out what exactly is missing but present in CPython. * We now have real CPython-ish hash-tables as dictionaries. Thanks Michael. * A nice feature (XXX mention it somewhere) is that on the PyPy prompt, Ctrl-C sends you to a CPython prompt in which you can play with PyPy at interp-level. All app-level names 'x' are available under 'w_x' names, i.e. the same name with a 'w_' prefix, and of course there is a 'space'. * Holger, Michael: operator module in pure Python. [However, Michael, let me point out that 'operator.add(2,3)' fails currently -- all of them do. staticmethods are not callable.] Sorry if I missed something above the "detail" level, or someone's due credits. This list was built by scanning through the svn checkin messages. A bientôt, Armin.

2 1

fun with functions
by Armin Rigo June 27, 2004

June 27, 2004

Hello, I'm reworking argument dispatching code, and I realize that we've always had a funny problem. In CPython, we have the following half-expected behavior: class X: def f(self, **kwds): print kwds x = X() x.f(v=5) -> {'v': 5} x.f(self=5) -> TypeError: multiple values for argument 'self' But of course, nothing wrong with: def f(**kwds): print kwds f(self=5) -> {'self': 5} However, this fails in PyPy with the same TypeError as above! The reason is that the call is done via the __call__ descriptor, which is a perfectly normal method: f.__call__ -> <method object at 0xe75fcac> (In CPython you get a <method-wrapper object>.) Now if it is a method we can investiate a bit: >>> f.__call__.im_self is f True >>> f.__call__.im_func <function object at 0xe73440c> >>> f.__call__.im_func.func_code <internal-code object at 0xe5fc98c> >>> f.__call__.im_func.func_code.co_varnames AttributeError: co_varnames but actually, even without (yet) exposing it at app-level, the code object has got a signature which says the function's argument list is (self, *args, **keywords). The name 'self' conflicts here with a 'self' keyword. Ack. It's nice and all to be able to inspect built-in functions' signature (which you cannot do yet, but it's just a matter of adding computed co_xxx attributes on this <internal-code object>)... but this is a problem. Moreover, we have a different compatibility problem too: in CPython, built-in functions don't have a __get__, so you can put them in classes and read them out without having the first argument bound to the instance. For example, if you put 'operator.add' in the class TrivialObjSpace, then 'space.add' is exactly 'operator.add' and not a bound version of it. What should we do about it? Maybe asking python-dev for guidance? Have 'unboundable function objects' -- like staticmethod, but the latter is not callable -- and make some carefully selected functions unboundable for compatibility? For the 1st problem we could force a general solution by saying that the argument that a method binds is hidden to keywords. It would also make the very first example of this e-mail work, although it doesn't work in CPython. A bientot, Armin.

3 3

Re: [pypy-dev] getting rid of multimethods?
by Laura Creighton June 26, 2004

June 26, 2004

There is a niceness about how the multimethods group everything in one place. Assuming we ever want to run more than one object space at the same time, or more than one representation of a given type, how would we structure things so that we don't just have 'a handful of stuff over here, and then some more from over there' ... ? Laura

4 3

AW: AW: [pypy-dev] AW: pypy-dev Digest, Vol 287, Issue 6
by "Jantzen, Günter" June 22, 2004

June 22, 2004

Anthony Baxter wrote: > Jantzen, Günter wrote: > > I am not a native speaker, so I don't understand the expression > > 'ass-backwards'. Seems not very polite to me. > > It's just a slightly stronger way of saying "the wrong way". > I'm Australian, and tend to use language like a blunt > instrument. <wink> Apologies for any offense. > Blunt is better than overly polite. :-)

1 0

AW: [pypy-dev] AW: pypy-dev Digest, Vol 287, Issue 6
by "Jantzen, Günter" June 22, 2004

June 22, 2004

> Anthony Baxter wrote: > Jantzen, Günter wrote: > > Holger Krekel wrote: > > > >>All i all, i think that we probably want to checkout joeq > >>some more. In particular, we might want to think about > >>trying to generate a QUAD representation of PyPy and reuse > >>some of the joeq infrastructure. We can expect to have less > >>integration issues than with the Low-Level-Virtual-Machine > >>(LLVM) project which is written in C++ (ever worked with > >>C++-libraries and dependencies? funfunfun). > >> > > > > > > C++ is an Industry standard based on C. > > C++ Libraries can be wrapped by C-Libraries (extern 'C') if > you have the source-code at hand. > > C++ Libraries that are not wrapped can contain native C++ > (for example class definitions). > > Then they can be called by C++ code only. It is not > possible to call > > C++ Libraries which are compiled with a different Compiler, because > > all C++ Implementations have different naming conventions > for methods > > and parameters(name mangling). > > > > I don't say LLVM is better for PyPy than joeq. But I think > it should > > be first considered what > > is right and second what is easy to implement. > > Huh? This is completely ass-backwards. Trying to implement it > in C++, merely because it's "an industry standard", is about > the worst reason in the world. And in your reply you point > out that the "industry standard" is a shambles when it comes > to dependencies and libraries. > > Consider what works, then worry about what's got some > supposed "industry standard" elephant stamp on it. If we > worried about "industry standards" we'd be all implementing > in ECMAScript, Java, and C#, _not_ Python. I did not suggest to implement something in C++ I only suggested to search for better reasons to make a decision. When choosing a library for a backend you will always have integration issues. It is never a fun to cross language borders. But as long as you wourk with 'industry standards' like Java, C or C++ you have a good chance to solve this problem one time and to encapsulate this stuff with a thin wrapper. For this reason it is more important to look from a design perspective. When choosing a backend, ask yourself if it will support the features which are your needs. I am not a native speaker, so I don't understand the expression 'ass-backwards'. Seems not very polite to me. regards Guenter

3 2

AW: [pypy-dev] AW: pypy-dev Digest, Vol 287, Issue 6
by "Jantzen, Günter" June 22, 2004

June 22, 2004

Holger Krekel wrote: > Well, to my ears an "industry standard" is usually the label > vendors put on their product. Moreover, with PyPy we cross > language borders all the time. It is inherent to the kind of > project we are doing. For practical matters, integration > issues and the "consider that works" principle are very > important to our kind of evolutionary development. Of course. My first mail was not very clear in this point. I said "industry standard", but what I meant was 'a reliable and well known language' in which we can trust so far that it is not necessary to discuss bitfiddling details in the first place. Scott Myers, author of 'Effective C++' titled one chapter 'Say what you mean; understand what you're saying' ;-)) I will try to be more clear for avoiding unecessary disaccord. > Anyway, i imagine that if we use joeq or LLVM as a backend we > might very well rewrite the most interesting parts in Python > at some point, depending on our experiences and the actual > development Sounds to me just like a somewhat stronger word for *forward* :-))) Cheers Guenter

1 0