Splitting RPython and PyPy
Hi Due to impressive work by Ronan Lamy, we're now able to split RPython and PyPy. Note that the fact of splitting this is not up to discussion, however, how we go about it is. During discussions with Armin we came up with the following plan: * We make a copy of pypy repo called rpython. it'll still live under pypy team on bitbucket. * We'll rename toplevel package of pypy to rpython, for the rpython part. Since we need to change ALL THE IMPORTS, we can do refactorings of imports now. Proposed changes: * to be moved from pypy to rpython: annotation, translator, rlib * pypy.config must be split somehow, same for tool, bin and for doc * move pypy.rpython namespace to rpython.rtyper * move pypy.rpython.lltypesystem to rpython.lltypesystem, same for ootype * pypy.rpython.memory becomes rpython.gc * pypy.objspace.flow becomes rpython.flowspace * testrunner and dotviewer can become independent packages * _pytest stays with pypy, however for rpython you can use whatever version of py.test you have installed. rpython test suite should also however be runnable under current contents of _pytest * RPython will come with setup.py and be a "normal" python package Cheers, fijal & armin
nice! rpython is a bit long for the top package name, the only other option I can think of would be rpyc (with c = compiler or chain) though which isn't that great; in any case translate.py should become either rpythonc or rpyc, in the tradition of cc and javac etc... I can even imagine a world/future where an rpython command alone would be for an interpreter for rpython.
* move pypy.rpython namespace to rpython.rtyper
I know that has been the historical name but the "r" in "rtyper" for restricted there is a bit of strange repetition with hindsight
Hi Maciej, Armin, On Sun, Oct 21, 2012 at 14:03 +0200, Maciej Fijalkowski wrote:
great stuff! Samuele's comments also make sense to me as well (rpython.typer etc.). Will there be a setup.py with pypy to automatically install perequisites (such as RPython, possibly other bits)? FWIW, I think it could help to eventually allow people more easily maintain modular rpython-level pypy modules by depending on a specific pypy version and declaring an entry-point in order to be automatically picked up by the pypy translation process. This way one could install multiple pypy-MODULE* packages, then cause a pypy build and all those modules would become part of the translated pypy-c. (happy to help with this and relate my experiences if wanted). It's not binary compatibility but would be a way to have a user choose modules without having to dive into translation target scripts etc. best, holger
On Sun, Oct 21, 2012 at 8:30 PM, holger krekel <holger@merlinux.eu> wrote:
IMO PyPy should not come with setup.py (unlike RPython). The main reason why is that PyPy is not a python package - after all python setup.py install will not do what you think it'll do. I think RPython will be the only dependency really.
On Mon, Oct 22, 2012 at 07:00 +0200, Maciej Fijalkowski wrote:
Trying "python setup.py install" can be made to bail out pointing to what one needs to do, though. Or maybe just install a "pypy_translate" script so one doesn't need to fish. The other orthogonal issue i raised is how to organise modularity of rpython-level pypy modules without the need to hack at the repo. But of course this could all be considered/done independently after the repo split so it's not directly related. best, holger
Le dimanche 21 octobre 2012 à 14:03 +0200, Maciej Fijalkowski a écrit :
I think that the changes should go deeper. I see model.py and operation.py as defining the flowgraph-based intermediate representation on which the rest of the tool chain operates. In contrast, the rest of pypy.objspace.flow only deals with translating Python callables to this IR. So I would suggest to move model.py and operation.py to rpython.flowgraph, and to also move there the various generic flow graph manipulation functions that are currently scattered around (e.g. in pypy.translator.backendopt.ssa).
* testrunner and dotviewer can become independent packages
+1 (well, I don't really have an opinion on testrunner). IMO, this implies that FunctionGraph should lose its view/show method.
Why the last stipulation? I would think that fishing out _pytest from pypy would require significant acrobatics once the repos are separated.
On Sun, Oct 21, 2012 at 12:35 PM, Antonio Cuni <anto.cuni@gmail.com> wrote:
We can always have a show() function which takes anything as a param, but anyways these are small details. Once things are split into two repositories it's easy to move stuff around within either one, it's just moving *between* the two of them that's hard, so let's make sure we get which repo everything lives in correctly, anything else is just a matter of `mv` + `sed` in a repository. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero
Hi Ronan, On Sun, Oct 21, 2012 at 9:28 PM, Ronan Lamy <ronan.lamy@gmail.com> wrote:
I believe you're putting too much focus on the flow object space; it's only part 1 of many in the translation process (even though it was that part that kept us from splitting RPython and PyPy so far). But anyway as others have pointed out we can discuss such additional moves after the main split.
That last stipulation was saying: if you install the PyPy and the RPython repos, then you should be able to use the (older) version of py.test that comes with PyPy in order to run tests in the RPython repo as well. In other words, RPython's tests should not suddenly become dependent on newer features of py.test. A bientôt, Armin.
nice! rpython is a bit long for the top package name, the only other option I can think of would be rpyc (with c = compiler or chain) though which isn't that great; in any case translate.py should become either rpythonc or rpyc, in the tradition of cc and javac etc... I can even imagine a world/future where an rpython command alone would be for an interpreter for rpython.
* move pypy.rpython namespace to rpython.rtyper
I know that has been the historical name but the "r" in "rtyper" for restricted there is a bit of strange repetition with hindsight
Hi Maciej, Armin, On Sun, Oct 21, 2012 at 14:03 +0200, Maciej Fijalkowski wrote:
great stuff! Samuele's comments also make sense to me as well (rpython.typer etc.). Will there be a setup.py with pypy to automatically install perequisites (such as RPython, possibly other bits)? FWIW, I think it could help to eventually allow people more easily maintain modular rpython-level pypy modules by depending on a specific pypy version and declaring an entry-point in order to be automatically picked up by the pypy translation process. This way one could install multiple pypy-MODULE* packages, then cause a pypy build and all those modules would become part of the translated pypy-c. (happy to help with this and relate my experiences if wanted). It's not binary compatibility but would be a way to have a user choose modules without having to dive into translation target scripts etc. best, holger
On Sun, Oct 21, 2012 at 8:30 PM, holger krekel <holger@merlinux.eu> wrote:
IMO PyPy should not come with setup.py (unlike RPython). The main reason why is that PyPy is not a python package - after all python setup.py install will not do what you think it'll do. I think RPython will be the only dependency really.
On Mon, Oct 22, 2012 at 07:00 +0200, Maciej Fijalkowski wrote:
Trying "python setup.py install" can be made to bail out pointing to what one needs to do, though. Or maybe just install a "pypy_translate" script so one doesn't need to fish. The other orthogonal issue i raised is how to organise modularity of rpython-level pypy modules without the need to hack at the repo. But of course this could all be considered/done independently after the repo split so it's not directly related. best, holger
Le dimanche 21 octobre 2012 à 14:03 +0200, Maciej Fijalkowski a écrit :
I think that the changes should go deeper. I see model.py and operation.py as defining the flowgraph-based intermediate representation on which the rest of the tool chain operates. In contrast, the rest of pypy.objspace.flow only deals with translating Python callables to this IR. So I would suggest to move model.py and operation.py to rpython.flowgraph, and to also move there the various generic flow graph manipulation functions that are currently scattered around (e.g. in pypy.translator.backendopt.ssa).
* testrunner and dotviewer can become independent packages
+1 (well, I don't really have an opinion on testrunner). IMO, this implies that FunctionGraph should lose its view/show method.
Why the last stipulation? I would think that fishing out _pytest from pypy would require significant acrobatics once the repos are separated.
On Sun, Oct 21, 2012 at 12:35 PM, Antonio Cuni <anto.cuni@gmail.com> wrote:
We can always have a show() function which takes anything as a param, but anyways these are small details. Once things are split into two repositories it's easy to move stuff around within either one, it's just moving *between* the two of them that's hard, so let's make sure we get which repo everything lives in correctly, anything else is just a matter of `mv` + `sed` in a repository. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero
Hi Ronan, On Sun, Oct 21, 2012 at 9:28 PM, Ronan Lamy <ronan.lamy@gmail.com> wrote:
I believe you're putting too much focus on the flow object space; it's only part 1 of many in the translation process (even though it was that part that kept us from splitting RPython and PyPy so far). But anyway as others have pointed out we can discuss such additional moves after the main split.
That last stipulation was saying: if you install the PyPy and the RPython repos, then you should be able to use the (older) version of py.test that comes with PyPy in order to run tests in the RPython repo as well. In other words, RPython's tests should not suddenly become dependent on newer features of py.test. A bientôt, Armin.
participants (7)
-
Alex Gaynor -
Antonio Cuni -
Armin Rigo -
holger krekel -
Maciej Fijalkowski -
Ronan Lamy -
Samuele Pedroni