Re: [pypy-dev] Support for __getitem__ in rpython?
On Tue, Dec 23, 2008 at 12:23 PM, Armin Rigo <arigo@tunes.org> wrote:
On Mon, Dec 15, 2008 at 08:47:26PM +0100, Hakan Ardo wrote:
cannot yet handle that the methods return NotImplemented. Would it be possible to handle that in a similar manner to how None is handled?
Not easily. The annotation framework of PyPy was never meant to handle the full Python language, but only a subset reasonable for writing
OK, that makes sens. On Tue, Dec 23, 2008 at 5:49 PM, Paolo Giarrusso <p.giarrusso@gmail.com> wrote:
And I also wonder if the RPython compiler can inline the __add__ call and optimize the tagging away.
That said, I do not know if what I'm suggesting is implementable in RPython, or if it would be a good idea. Just my 2 cents, since this might be what Hakan is looking for.
Yes, in most cases of interest, the annotator can determined whether the __add__ method always/never return NotImplemented and thus the test will be removed. If we'll not be able to handle the most general case where the choise between __add__ and __radd__ has to be done at runtime, that's fine. If I use None instead of NotImplemented I get the behaviour I want. The following code does for example compile: class mystr(object): def __init__(self,s): self.s=str(s) def __str__(self): return self.s def __add__(self,other): return mystr(self.s+str(other)) def __radd__(self,other): return mystr(str(other)+self.s) __add__._annspecialcase_ = 'specialize:argtype(1)' __radd__._annspecialcase_ = 'specialize:argtype(1)' class pair(object): def __init__(self,a,b): self.a=a self.b=b def __str__(self): return "(%d,%d)"%(self.a,self.b) def __add__(self,other): if isinstance(other,pair): return pair(self.a+other.a, self.b+other.b) else: return None __add__._annspecialcase_ = 'specialize:argtype(1)' __radd__=__add__ def dotst_notimplemented(): a=mystr('a') b=pair(1,2) return (str(b+a), str(a+b)) If I use the following helper to call the method: def do_add_radd(lop,rop): if isinstance(rop,lop.__class__) and not isinstance(lop,rop.__class__): r=rop.__radd__(lop) if r is None: return lop.__add__(rop) return r else: r=lop.__add__(rop) if r is None: return rop.__radd__(lop) return r But if I replace None with NotImplemented, it does not compile anymore. So, can we get the annotater to treat NotImplemented in a similar manner as it treats None? -- Håkan Ardö
Hi, On Fri, Dec 26, 2008 at 11:31:53AM +0100, Hakan Ardo wrote:
But if I replace None with NotImplemented, it does not compile anymore. So, can we get the annotater to treat NotImplemented in a similar manner as it treats None?
You can try, but it's messy. It's not a problem for the annotator but for the later RTyper. None is implemented as a NULL pointer by the RTyper; I don't know how you would distinguish between None-as-a-NULL and NotImplemented. You could try to go for something like ((TYPE*) 1), but this doesn't work on top of ootype, where you really have only one NULL value. A bientot, Armin.
On 27/12/2008, Armin Rigo <arigo@tunes.org> wrote:
Hi,
On Fri, Dec 26, 2008 at 11:31:53AM +0100, Hakan Ardo wrote:
But if I replace None with NotImplemented, it does not compile anymore. So, can we get the annotater to treat NotImplemented in a similar manner as it treats None?
You can try, but it's messy. It's not a problem for the annotator but for the later RTyper. None is implemented as a NULL pointer by the RTyper; I don't know how you would distinguish between None-as-a-NULL and NotImplemented. You could try to go for something like ((TYPE*) 1), but this doesn't work on top of ootype, where you really have only one NULL value.
I don't know ootype, but why not having a NotImplemented singleton type, and returning a pointer to its instance? Python has a None singleton type as well, so it makes sense. Regards -- Paolo Giarrusso
Hi Paolo, On Sat, Dec 27, 2008 at 07:56:51PM +0100, Paolo Giarrusso wrote:
I don't know ootype, but why not having a NotImplemented singleton type, and returning a pointer to its instance? Python has a None singleton type as well, so it makes sense.
See our doc: http://codespeak.net/pypy/dist/pypy/doc/translation.html and http://codespeak.net/pypy/dist/pypy/doc/rtyper.html . A bientot, Armin.
On Sat, Dec 27, 2008 at 7:45 PM, Armin Rigo <arigo@tunes.org> wrote:
You can try, but it's messy. It's not a problem for the annotator but for the later RTyper. None is implemented as a NULL pointer by the
Atatched is a small patch for the annotator that makes it treat None and NotImplemented alike. This is all that is needed for most cases as all NotImplemented are typically removed by the optimisations performed by the annotator. At http://hakan.ardoe.net/pypy/ I have placed special_methods.py that adds support for the methods listed below together with 42 test including the relevant parts of test_augassign.py and test_binop.py from the cpython source (somewhat modified to work). The methods currently supported are: __str__, __repr__, __len__, __getitem__, __setitem__, __add__, __mul__, __sub__, __div__, __floordiv__, __mod__, __xor__, __rshift__, __lshift__, __radd__, __rmul__, __rsub__, __rdiv__, __rfloordiv__, __rmod__, __rxor__, __rrshift__, __rlshift__, __iadd__, __imul__, __isub__, __idiv__, __ifloordiv__, __imod__, __ixor__, __irshift__, __ilshift__ With this implementation, the opperation str(o) calls o.__str__(), but the opperation "%s"%o does not. I don't know why. -- Håkan Ardö
On Mon, Dec 29, 2008 at 16:15, Hakan Ardo <hakan@debian.org> wrote:
On Sat, Dec 27, 2008 at 7:45 PM, Armin Rigo <arigo@tunes.org> wrote:
You can try, but it's messy. It's not a problem for the annotator but for the later RTyper. None is implemented as a NULL pointer by the
Atatched is a small patch for the annotator that makes it treat None and NotImplemented alike. This is all that is needed for most cases as all NotImplemented are typically removed by the optimisations performed by the annotator. That can be made to work, but if such a method returns None you get completely different semantics (trying again with something else) from CPython (which will maybe return a failure, or return None for the result of such an operation), so you have to restrict the allowed semantics in RPython to the "most cases" you are referring to.
Basically, I'd propose that in RPython with your patch, those methods (__add__ etc.) cannot return None (I can't think of a possible use case for an addition returning None), while I guess NotImplemented can only be returned by them, and not by any other function or method (do the current sources ever use NotImplemented?). There must be no function which can choose to return None or NotImplemented. Regards -- Paolo Giarrusso
On Mon, Dec 29, 2008 at 8:55 PM, Paolo Giarrusso <p.giarrusso@gmail.com> wrote:
Atatched is a small patch for the annotator that makes it treat None and NotImplemented alike. This is all that is needed for most cases as all NotImplemented are typically removed by the optimisations performed by the annotator.
That can be made to work, but if such a method returns None you get completely different semantics (trying again with something else) from CPython (which will maybe return a failure, or return None for the result of such an operation), so you have to restrict the allowed
No, the patch do distinguish between None and NotImplemented. What I mean is that NotImplemented is treated in a similar manner as to how None is treated. The following crazy construction do compile and generate the same result as in cpython ('OK', 'String', 'None', 'None'): class mystr: def __init__(self,s): self.s=s def __str__(self): return self.s def __add__(self,other): if isinstance(other,mystr): return NotImplemented s=self.s+other if s=='None': return None else: return s __add__._annspecialcase_ = 'specialize:argtype(1)' def __radd__(self,other): return str(other)+self.s __radd__._annspecialcase_ = 'specialize:argtype(1)' def dotst_nonestr(): s1=mystr('No')+'ne' if s1 is None: s1='OK' s2=mystr('Str')+'ing' s3=mystr('No')+mystr('ne') s4='No'+mystr('ne') return (s1,s2,s3,s4) -- Håkan Ardö
participants (3)
-
Armin Rigo -
Hakan Ardo -
Paolo Giarrusso