Hi all, I am investigating how to make virtualenv working on pypy and I'm running into a couple of issues: the most important one is that virtualenv relies on sys.prefix (which does not exists in pypy) to find the standard library, and the other is that the standard library of pypy is supposed to be put in /usr/share instead of /usr/lib (or /usr/local/*). Currently, a pypy installation is supposed to have this structure: /usr/bin/pypy-c /usr/share/pypy-1.2/pypy/lib/ /usr/share/pypy-1.2/lib-python/modified-2.5.2 /usr/share/pypy-1.2/lib-python/2.5.2 In such a situation, sys.pypy_prefix is set to '/usr/share/pypy-1.2'. I propose to change it in this way: /usr/bin/pypy-c /usr/lib/pypy1.2/lib-pypy/ /usr/lib/pypy1.2/lib-python/modified-2.5.2 /usr/lib/pypy1.2/lib-python/2.5.2 where lib-pypy contains what it's now in pypy/lib. In such a situation, sys.prefix would be set to '/usr', in a similar way as cpython. Also, we should also add a sys.exec_prefix which is meant to be always equal to sys.prefix (at least for now). (I removed the dash in pypy-1.2 for consistency with cpython, which uses something like lib/python2.6). Moreover, I would also like virtualenv to work from an svn checkout/source tarball of pypy, without any needing of installing it system-wide. To do so, we need to find a sensible value for sys.prefix, considering that tools like virtualenv suppose to find the stdlib under sys.prefix+'lib/'+something_else. So, the proposed new structure is this: /path/to/pypy-trunk/ /path/to/pypy-trunk/lib/pypy1.2/{lib-pypy,modified-2.5.2,2.5.2} sys.prefix == '/path/to/pypy-trunk' The drawback is that before getting to the real files you have to walk a lot of empty directories, and that we should manually change the name of pypy1.2 each time we increase the version number (not that it happens very often :-)). One side-advantage is that in this way we would move pypy/lib outside the main pypy package, which is good because it's not really a part of it. Finally, we should probably think of where to put the include/ directory (plus other that might be needed to build extension), but I'll let cpyext experts to say what it's better :-) What do you think? Any comment/suggestion/problem that I overlooked? ciao, Anto
Hey anto.
I expressed my concerns already, but I'm going to express them once
again, since amaury also said that on IRC.
I really dislike having version number hardcoded in source checkout
for a whole variety of reasons. I don't think need to have virtualenv
working from source checkout is enough to push that through.
How about install script that does that maybe?
Cheers,
fijal
On Wed, May 26, 2010 at 8:55 AM, Antonio Cuni
Hi all,
I am investigating how to make virtualenv working on pypy and I'm running into a couple of issues: the most important one is that virtualenv relies on sys.prefix (which does not exists in pypy) to find the standard library, and the other is that the standard library of pypy is supposed to be put in /usr/share instead of /usr/lib (or /usr/local/*).
Currently, a pypy installation is supposed to have this structure: /usr/bin/pypy-c /usr/share/pypy-1.2/pypy/lib/ /usr/share/pypy-1.2/lib-python/modified-2.5.2 /usr/share/pypy-1.2/lib-python/2.5.2
In such a situation, sys.pypy_prefix is set to '/usr/share/pypy-1.2'.
I propose to change it in this way: /usr/bin/pypy-c /usr/lib/pypy1.2/lib-pypy/ /usr/lib/pypy1.2/lib-python/modified-2.5.2 /usr/lib/pypy1.2/lib-python/2.5.2
where lib-pypy contains what it's now in pypy/lib. In such a situation, sys.prefix would be set to '/usr', in a similar way as cpython. Also, we should also add a sys.exec_prefix which is meant to be always equal to sys.prefix (at least for now). (I removed the dash in pypy-1.2 for consistency with cpython, which uses something like lib/python2.6).
Moreover, I would also like virtualenv to work from an svn checkout/source tarball of pypy, without any needing of installing it system-wide. To do so, we need to find a sensible value for sys.prefix, considering that tools like virtualenv suppose to find the stdlib under sys.prefix+'lib/'+something_else.
So, the proposed new structure is this:
/path/to/pypy-trunk/ /path/to/pypy-trunk/lib/pypy1.2/{lib-pypy,modified-2.5.2,2.5.2} sys.prefix == '/path/to/pypy-trunk'
The drawback is that before getting to the real files you have to walk a lot of empty directories, and that we should manually change the name of pypy1.2 each time we increase the version number (not that it happens very often :-)). One side-advantage is that in this way we would move pypy/lib outside the main pypy package, which is good because it's not really a part of it.
Finally, we should probably think of where to put the include/ directory (plus other that might be needed to build extension), but I'll let cpyext experts to say what it's better :-)
What do you think? Any comment/suggestion/problem that I overlooked?
ciao, Anto _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Hi, I also don't like too much to have a hardcoded version number, but when I asked for alternatives nobody suggested anything. On IRC, amaury suggested to install the whole pypy distribution in its self-contained directory, more or less as cpython does on windows. I didn't think about this solution, but now I see that it might be a good one, as it would allow to have the same hierarchy on svn checkouts, user-specific installations and system-wide installations. The drawback is that it's a bit non-standard on unix; moreover, if we install pypy in say /opt/pypy1.2, it would be hard to put a binary in /usr/bin without hardcoding the path to pypy1.2 somewhere. So, for me there are four possible solutions: 1) leave things as they are on branch/sys-prefix and have the version number hardcoded in svn 2) put the whole distribution in its own directory, as jython or cpython on windows. This has the open problem of determining where the directory is, as described above 3) don't hardcode the version number in svn, but add a special case to virtualenv to detect if we are inside a pypy checkout to handle it specially 4) don't care about running virtualenv inside a pypy checkout Personally, I would exclude (4): I think that it would be very cool for people to try pypy in a sandboxed virtualenv without having to install it, and it would be useful to us too. So, before I do more work, I'd like to hear what people think and which of the alternatives they prefer. ciao, Anto On 18/06/10 19:01, Maciej Fijalkowski wrote:
Hey anto.
I expressed my concerns already, but I'm going to express them once again, since amaury also said that on IRC.
I really dislike having version number hardcoded in source checkout for a whole variety of reasons. I don't think need to have virtualenv working from source checkout is enough to push that through.
How about install script that does that maybe?
Cheers, fijal
Hi Anto, On Sat, Jun 19, 2010 at 10:29:58AM +0200, Antonio Cuni wrote:
The drawback is that it's a bit non-standard on unix; moreover, if we install pypy in say /opt/pypy1.2, it would be hard to put a binary in /usr/bin without hardcoding the path to pypy1.2 somewhere.
No, we can put a symlink in /usr/bin going to /opt/pypy1.2/bin/pypy. I have seen a few projects do that and it would just work (the current logic to search for the pypy directory follows symlinks). A bientot, Armin.
Hi,
2010/6/19 Antonio Cuni
I also don't like too much to have a hardcoded version number, but when I asked for alternatives nobody suggested anything.
On IRC, amaury suggested to install the whole pypy distribution in its self-contained directory, more or less as cpython does on windows. I didn't think about this solution, but now I see that it might be a good one, as it would allow to have the same hierarchy on svn checkouts, user-specific installations and system-wide installations.
The drawback is that it's a bit non-standard on unix; moreover, if we install pypy in say /opt/pypy1.2, it would be hard to put a binary in /usr/bin without hardcoding the path to pypy1.2 somewhere.
Is it possible to just put a symlink, or a small script in /usr/bin? -- Amaury Forgeot d'Arc
On 19/06/10 11:00, Amaury Forgeot d'Arc wrote:
The drawback is that it's a bit non-standard on unix; moreover, if we install pypy in say /opt/pypy1.2, it would be hard to put a binary in /usr/bin without hardcoding the path to pypy1.2 somewhere.
Is it possible to just put a symlink, or a small script in /usr/bin?
yes, the symlink should be possible, as armin also points out. I already thought about it, but I was not sure that distributions like ubuntu allows to put a symlink in /usr/bin to something external. But indeed, looking at my /usr/bin it seems that symlinks are used a lot, so it should be fine. So, does this mean that we are going for solution number 2?
Personally, if given such choice I would go with 4) (don't care about
virtualenv from source checkout). I don't care myself and I don't
think many people would care (since they'll get pypy installed from
say ubuntu or fedora most likely).
Cheers,
fijal
On Sat, Jun 19, 2010 at 3:07 AM, Antonio Cuni
On 19/06/10 11:00, Amaury Forgeot d'Arc wrote:
The drawback is that it's a bit non-standard on unix; moreover, if we install pypy in say /opt/pypy1.2, it would be hard to put a binary in /usr/bin without hardcoding the path to pypy1.2 somewhere.
Is it possible to just put a symlink, or a small script in /usr/bin?
yes, the symlink should be possible, as armin also points out. I already thought about it, but I was not sure that distributions like ubuntu allows to put a symlink in /usr/bin to something external. But indeed, looking at my /usr/bin it seems that symlinks are used a lot, so it should be fine.
So, does this mean that we are going for solution number 2?
participants (4)
-
Amaury Forgeot d'Arc -
Antonio Cuni -
Armin Rigo -
Maciej Fijalkowski