Hi everyone I am currently seriously confused about the notion of RPython, and I hope someone can show me the light here. ;) I guess my basic problem is: How do I find out if some code is RPython? And especially in the context of the array module im currently working on, how do I find out if a module is RPython? One of my working hypothesis was that whatever can be geninterped is RPython. But by that definition, a *lot* more comes out as RPython than the documentation says. Is it just that the documentation is out-of-date or is geninterping not a good criterion? E.g., when Christian states that _codecs is now RPython, does that just mean it can be geninterped or is there something else to it? Armin just had a checkin where he replaced str.isalpha() with something else because string methods are not RPython. But I can geninterp and even C compile string methods just fine. Did he use the term "RPython" loosely, meaning something like "what can be usefully annotated" (ie, string methods always result in SomeObject, that's probably why they're frowned upon in the interpreter code)? Cheers Nik
Niklaus Haldimann <nhaldimann@gmx.ch> writes:
Hi everyone
I am currently seriously confused about the notion of RPython, and I hope someone can show me the light here. ;)
I guess my basic problem is: How do I find out if some code is RPython?
Hrm. If it gets annotated without SomeObjects, I guess it's RPython.
Armin just had a checkin where he replaced str.isalpha() with something else because string methods are not RPython. But I can geninterp and even C compile string methods just fine.
Are they C compiled to code using the Python C API? Cheers, mwh -- Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it. -- spaf (1992)
Hi Niklaus,
I am currently seriously confused about the notion of RPython, and I hope someone can show me the light here. ;)
blink blink :)
I guess my basic problem is: How do I find out if some code is RPython? And especially in the context of the array module im currently working on, how do I find out if a module is RPython?
I see you want to write the array module at interplevel. This makes sense, applevel has problems here. interplevel with possibly give you problems as well, I guess, because I have no clue how to express the buffer interface for instance. We don't have pointers and too few basic types. If you stubleofver such problems, please don't hesitate to contact the list.
One of my working hypothesis was that whatever can be geninterped is RPython. But by that definition, a *lot* more comes out as RPython than the documentation says. Is it just that the documentation is out-of-date or is geninterping not a good criterion? E.g., when Christian states that _codecs is now RPython, does that just mean it can be geninterped or is there something else to it?
Well, I think my notation is a biut confusing, of course. Should have tagged nn-geninterpable code differently. The tag "NOT_RPYTHON" is used inside PyPyto signal stuff that we should not try to translate, the annotator will raise an exception if it sees such a thing, and so on. So I used the same tag to signal that "This applevel code is not suitable for geninterp". Of course, this is some kind of "applevel RPython" in the sense that it can be pured through geninterp to create an RPython interplevel module. And this is all what it is saying. If you are writing on interp level, geninterp doesn't help. You need to translate to some target and see whether flowspace and rtyper are happy with it.
Armin just had a checkin where he replaced str.isalpha() with something else because string methods are not RPython. But I can geninterp and even C compile string methods just fine. Did he use the term "RPython" loosely, meaning something like "what can be usefully annotated" (ie, string methods always result in SomeObject, that's probably why they're frowned upon in the interpreter code)?
I used it loosely, in a sense. Maybe I should rename the tag. cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Christian Tismer wrote:
I see you want to write the array module at interplevel. This makes sense, applevel has problems here. interplevel with possibly give you problems as well, I guess, because I have no clue how to express the buffer interface for instance. We don't have pointers and too few basic types. If you stubleofver such problems, please don't hesitate to contact the list.
I am actually trying to write it in applevel, making sure it can be geninterped. If I can do this in a resonable amount of time I might try to rewrite some stuff on interplevel. You can see the current effort here, BTW: http://codespeak.net/svn/user/nik/array_py/trunk/src/array.py I think the buffer interface is a no-brainer, but maybe I'm missing something. Our buffer type is written at applevel, we have no obligation at all to emulate CPython's C-based buffer interface. We can define our own interface, and in fact the current buffer implementation sort of does that. Hacking array support into the current buffer will be a matter of 2-3 lines of code, AFAICT. More effort would be a waste of time anyway, as buffer is as good as deprecated. ;)
Well, I think my notation is a biut confusing, of course. Should have tagged nn-geninterpable code differently. The tag "NOT_RPYTHON" is used inside PyPyto signal stuff that we should not try to translate, the annotator will raise an exception if it sees such a thing, and so on. So I used the same tag to signal that "This applevel code is not suitable for geninterp". Of course, this is some kind of "applevel RPython" in the sense that it can be pured through geninterp to create an RPython interplevel module.
I see, that really clears things up a bit. Is it correct to say that geninterping such an "applevel RPython" module guarantees that the resulting interplevel code can in fact be translated? That sounds a bit too much like magic to me.
And this is all what it is saying. If you are writing on interp level, geninterp doesn't help. You need to translate to some target and see whether flowspace and rtyper are happy with it.
OK. Just to be a bit more explicit about this, I do something like:
t = Translator(my_rpython_function) t.annotate([int])
And check that the inferred return type is not SomeObject. This basically means that flowspace and rtyper are happy, right? What I don't understand ATM is where the translator will get the above "[int]" annotation hint from in the real world ... Thanks for showing me the light so far. ;) Cheers Nik
Niklaus Haldimann wrote: ...
I think the buffer interface is a no-brainer, but maybe I'm missing something. Our buffer type is written at applevel, we have no obligation at all to emulate CPython's C-based buffer interface. We can define our own interface, and in fact the current buffer implementation sort of does that. Hacking array support into the current buffer will be a matter of 2-3 lines of code, AFAICT. More effort would be a waste of time anyway, as buffer is as good as deprecated. ;)
Sounds reasonable. Actually I didn't look too closely into the array module myself, I was just repeating what another guy reported when trying to implement it. So it might be easier than I expcetged. ...
I see, that really clears things up a bit. Is it correct to say that geninterping such an "applevel RPython" module guarantees that the resulting interplevel code can in fact be translated? That sounds a bit too much like magic to me.
No, it doesn't. It tells you that the flow space is happy with your code, but still there may be issues with annotation I think. And stuff like producing an infinite recursion is not detected at all. For instance, we had code that relied on applevel in a way that the implementation called into the interpreter for something that then goes back to the implementation which... So I'd say geninterping is a necessary but not sufficient condition. ...
OK. Just to be a bit more explicit about this, I do something like:
t = Translator(my_rpython_function) t.annotate([int])
And check that the inferred return type is not SomeObject. This basically means that flowspace and rtyper are happy, right? What I don't understand ATM is where the translator will get the above "[int]" annotation hint from in the real world ...
You are giving it the hint to have some reasonable starting point for testing, assuming that this assumption generally holds. In the real world, everything and all of PyPy is run through the annotator, which re-iterates stuff until it ends up with a proven interface, or it has to give up with someobjectness. So the realanalysis depends on whether your implementation is really called in the right way, all over PyPy. ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Christian Tismer wrote:
I see, that really clears things up a bit. Is it correct to say that geninterping such an "applevel RPython" module guarantees that the resulting interplevel code can in fact be translated? That sounds a bit too much like magic to me.
No, it doesn't. It tells you that the flow space is happy with your code, but still there may be issues with annotation I think. And stuff like producing an infinite recursion is not detected at all. For instance, we had code that relied on applevel in a way that the implementation called into the interpreter for something that then goes back to the implementation which...
So I'd say geninterping is a necessary but not sufficient condition.
Thanks for the pointers Christian, I see much clearer now. The strategy for the array module is now set (also after an email exchange with Armin): I'm writing it in applevel, not even caring much about geninterping it. It's no use thinking about an interplevel RPython version right now, because it's hard (e.g., it would require some serious hacks to implement a method like array.append which accepts several data types) and will potentially get easier in the future with more low-level types available. Cheers Nik
participants (3)
-
Christian Tismer -
Michael Hudson -
Niklaus Haldimann