Hi Armin, all, I've put quite some work into genrpy.py in the last few weeks. It seems to produce usefulcode, although it could of course be nicer with some more effort. The speed effect appears to be somewhere around 40% over interpreted appspace code, not really disappointing, but also not really overwhelming. The real effect of such translation lies probably in opening things to further optimization in a later code generation step. The code works fine with _formatting.py, md5.py and strutil.py. These all appear to obey RPython restrictions, as probably many others. There are some enhancements possible: Right now, I'm using the flow space as it is, but I do not try to create specialized functions for constant arguments. Flow space provides some way to do this, but translator doesn't allow it. I'm thinking to play with this. It would make much sense in the md5 module, where many small functions with constant arguments could be curryfied. In effect, with some more support, the flowspace could identify more constants than it does now. The way I'm creating the function interfaces is working, but less than optimum. I stupidly followed the pattern that Armin used for C code generation. Later on I figured out that I could create functions with names and default arguments directly. Armin, I'm referring to the distinction of f_<funcname> and fastf_<funcname>. Instead of using the pattern def f_<funcname>(space, *args_w) and later plumbing the defaults in, I could have done it in the function header, directly. I will enhance this later, provided that we want to use this module at all. How do we want to integrate this stuff? First I think it needs a different name. Then, there is the question whether there should be one huge module generated from the union of all app-space implementations, or if there should be onefile for every app-spacemodule part? I thought of acontainerlike pypy.appspace.generated where all stuff could be accumulated. Do we want togenerate stuff on demand, based upon modification time of modules, or better do it by hand? Another questio is if it is so good to loose all the local names early,or if I should try to regenerate local names wherever possible? Another thing I was considering is not to create all these goto blocks, but to turn them into simple functions. This would give us massive recursion, but if we add a tail recursion feature to pypy, this would show up quite nicely. It would make the source shorter and would really allow to use given local names, because each code block has its own closure, then. I have done a lot of local tests. The test code is still sitting in the module and should go elsewhere. I did not yet tryto integrate it into the app space calls of stdobjspace. This is easy to do,but I need consensus how wewant this to happen. Please review. You can select a few test functions by changing line 1084. For the current main function, you need to create an importable appspace/generated folder. You also will needto change the temp output file for your needs, at the moment this is "d:/tmp/look.py" :-) I will supply a real entrypoint and a cleaner interface when I have some feedback. all the best -- chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Christian Tismer <tismer@stackless.com> writes:
Hi Armin, all,
I've put quite some work into genrpy.py in the last few weeks. It seems to produce usefulcode, although it could of course be nicer with some more effort.
The speed effect appears to be somewhere around 40% over interpreted appspace code, not really disappointing, but also not really overwhelming.
The real effect of such translation lies probably in opening things to further optimization in a later code generation step.
The code works fine with _formatting.py, md5.py and strutil.py. These all appear to obey RPython restrictions, as probably many others.
Hmm! Could we use this to make an interpreter-level implementation of the argument parsing code from a nice clear application level implementation? Or does this still result in hideous circularities? Probably. Oh well. Cheers, mwh -- Lisp does badly because we refuse to lie. When people ask us if we can solve insoluble problems we say that we can't, and because they expect us to lie to them, they find some other language where the truth is less respected. -- Tim Bradshaw, comp.lang.lisp
Michael Hudson wrote: ...
Hmm! Could we use this to make an interpreter-level implementation of the argument parsing code from a nice clear application level implementation? Or does this still result in hideous circularities? Probably. Oh well.
I think we could try this. We can look at circularities quite easily. Just for fun, I have run genrpy twice over a function. That creates the whole interpreter as a nice single huge Python source file. Although this doesn't make too much sense for producing the 'real' code, it procides a new way to do source analysis. You know that everything is in the file, the whole interpreter, you can single step, add debug calls, whatsoever, without having to hack around in the real sources. well, I'll continue with this. Would like to run the huge file. At the moment, I have real trouble with old-style class instances. No idea how to create them, they seem to be not correctly wrapped. cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
On Tue, Jan 11, 2005 at 11:31 +0000, Michael Hudson wrote:
Christian Tismer <tismer@stackless.com> writes:
Hi Armin, all,
I've put quite some work into genrpy.py in the last few weeks. It seems to produce usefulcode, although it could of course be nicer with some more effort.
The speed effect appears to be somewhere around 40% over interpreted appspace code, not really disappointing, but also not really overwhelming.
The real effect of such translation lies probably in opening things to further optimization in a later code generation step.
The code works fine with _formatting.py, md5.py and strutil.py. These all appear to obey RPython restrictions, as probably many others.
Hmm! Could we use this to make an interpreter-level implementation of the argument parsing code from a nice clear application level implementation? Or does this still result in hideous circularities? Probably. Oh well.
We decided that we had enough of hideous circularities in this area some time ago :-) So we don't have app-level code for argument parsing anymore, i think. It turned out that the interp-level code wasn't really that bad (in interpreter/argument.py). cheers, holger
hpk@trillke.net (holger krekel) writes:
We decided that we had enough of hideous circularities in this area some time ago :-) So we don't have app-level code for argument parsing anymore, i think. It turned out that the interp-level code wasn't really that bad (in interpreter/argument.py).
Yes, but that arguably isn't RPythonic in the way it uses dictionaries. Cheers, mwh -- This makes it possible to pass complex object hierarchies to a C coder who thinks computer science has made no worthwhile advancements since the invention of the pointer. -- Gordon McMillan, 30 Jul 1998
Hi Christian, Very nice work !
First I think it needs a different name.
geninterplevel.py maybe ..? Here's some more feedback on the questions as you put them in the source...
- do we want to auto-generate stuff? - do we wantamoduleperapp-spaceoperation?
We should probably auto-generate the stuff, using the py.tool.cache mecanisms. If we do so, then the files we generate should have a corresponding granularity. I think that we can easily list all app-level helpers of the same module (gateways know each others to be able to call each others). Then we could translate one-module-at-a-time. With md5 checksums or whatever :-)
- do we want to create code that is more similar to the app code?
I have mixed feelings about this. My initial reaction is "no" but then there are use cases for the object space knowing a bit more about the names that the interpreter gives to the objects... for example, for a debugging "history object space" it would be convenient to record the movements within the local variables too. Not sure how to do that or if the same thing can be reused for the flow object space.
- do we want to create specialized code for constants? - do we want to inline small functions?
I'd say no. That's the job of a future optimization pass.
- do we want to use small tail-functions instead of goto?
Ack. You want to blow CPython up or force us to use Stackless to test genrpy :-) But clearly, the ability to generate such code will be nice in the future.
- do we want to translate non-rpythonic code as well?
Tough one :-) Maybe, using non-constant global dicts... but we need anyway now to pin down precisely all simplifying assumptions that the flow object space is based on, and document this as R-Python. This can be done independently of the type-inferencing assumptions which are still in an fuzzy state, but not related to genrpy. A bientot, Armin.
participants (4)
-
Armin Rigo -
Christian Tismer -
hpk@trillke.net -
Michael Hudson