[PyPy-issue] [issue614] Packaging PyPy: installation path for PyPy's libraries
Dave Malcolm
pypy-dev-issue at codespeak.net
Thu Dec 23 02:54:02 CET 2010
New submission from Dave Malcolm <dmalcolm at redhat.com>:
Short version:
I'm working on packaging PyPy in RPM form for Fedora; see:
https://bugzilla.redhat.com/show_bug.cgi?id=588941
and ran into an issue with how the pypy binary searches for its libraries.
I'm attaching a patch I've written that fixes things, though it's not good enough
yet to be applied to hg.
Long version:
As I understand things, PyPy's prebuilt tarballs have a directory structure
generated by pypy/tool/release/package.py.
Looking at e.g.
http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
everything is below:
pypy-1.4.1-linux64/
with:
pypy-1.4.1-linux64/lib_pypy/
pypy-1.4.1-linux64/lib-python/
pypy-1.4.1-linux64/lib-python/2.5.2/*.py (and subdirs)
pypy-1.4.1-linux64/lib-python/modified-2.5.2/*.py (and subdirs)
pypy-1.4.1-linux64/site-packages/*.py
pypy-1.4.1-linux64/bin/pypy (the JIT binary)
pypy-1.4.1-linux64/include/ (some header files)
(See also pypy/tool/release/test/test_package.py, which verifies this structure)
This works for a standalone distribution of PyPy.
However, eventually Linux distribution will want to ship "system" builds of PyPy,
deployed as .rpmor .deb packages. Those packages ideally ought to follow the layout
recommendations of the Filesystem Hierarchy Standard:
http://www.pathname.com/fhs/
In particular this suggests separating out programs (to /usr/bin) from libraries (to
/usr/lib) and data (/usr/share).
Exactly where the pypy libraries should go is debatable; they're all architecture-
independent for now, but I hope we'll soon have a rich collection of extension
modules that are stable enough to package.
So I'm attempting to split the libraries out to /usr/lib (and /usr/lib64 on 64-bit,
which is a Fedora-ism, AIUI).
My naive attempt to do this leads to a /usr/bin/pypy which emits this message at
startup:
$ pypy-jit
debug: WARNING: library path not found, using compiled-in sys.path
'import site' failed
Python 2.5.2 (78826, Dec 15 2010, 01:41:24)
[PyPy 1.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
debug: OperationError:
debug: operror-type: ImportError
debug: operror-value: No module named _pypy_interact
and has this sys.path:
$ pypy -c 'import sys; print sys.path'
debug: WARNING: library path not found, using compiled-in sys.path
'import site' failed
['', '/builddir/build/BUILD/pypy-1.4/lib_pypy',
'/builddir/build/BUILD/pypy-1.4/lib-python/modified-2.5.2',
'/builddir/build/BUILD/pypy-1.4/lib-python/2.5.2',
'/builddir/build/BUILD/pypy-1.4/lib-python/2.5.2/plat-linux2']
pypy/translator/goal/app_main.py's startup code calls into sys's state.py, which
has:
def getinitialpath(prefix):
which searches for the dirs, raising OSError if they're not found. If they're all
their, it returns them in a list.
app_main wraps this as sys.pypy_initial_path, turning OSError into a return of None.
I'm attaching a proof-of-concept patch which adds an initial search for the
directory structure, this time within LIBRARY_INSTALLATION_PATH.
Ideally this would be an optional, configuration-supplied constant, but for now,
this is a non-existant variable, which I pre-process within the build, using sed,
thus:
sed -i \
-e 's|LIBRARY_INSTALLATION_PATH|"%{pypyprefix}"|' \
pypy/translator/goal/app_main.py
where %{pypyprefix} is expanded by another tool, giving the string literal
"/usr/lib64/pypy-1.4.1"
I wanted to prove the concept, and hour-long builds make for tedious experimentation
:(
Presumably this should be something like a:
--library-installation-path=/usr/lib
that gets passed to the translate.py
Building with this patch (with the above caveat), I obtain a binary that, when
copied to /usr/bin/pypy, successfully finds its libraries within our downstream
distribution's directory structure (under /usr/lib64/pypy-1.4.1 on this build):
$ pypy
Python 2.5.2 (, Dec 22 2010, 23:14:02)
[PyPy 1.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``PyPy is an exciting technology
that lets you to write fast, portable, multi-platform interpreters with less
effort''
>>>> import sys
>>>> sys.path
['', '/usr/lib64/pypy-1.4.1/lib_pypy', '/usr/lib64/pypy-1.4.1/lib-python/modified-
2.5.2', '/usr/lib64/pypy-1.4.1/lib-python/2.5.2', '/usr/lib64/pypy-1.4.1/lib-
python/2.5.2/plat-linux2', '/usr/lib64/pypy-1.4.1/site-packages']
and also successfully finds its libraries within the build tree, when run from there
in the usual "goal" subdirectory.
>>>> import collections
>>>> collections.__file__
'/usr/lib64/pypy-1.4.1/lib_pypy/collections.pyc'
----------
effort: ???
files: pypy-1.4.1-add-LIBRARY_INSTALLATION_PATH.patch
messages: 2031
nosy: dmalcolm, pypy-issue
priority: feature
release: ???
status: unread
title: Packaging PyPy: installation path for PyPy's libraries
_______________________________________________________
PyPy development tracker <pypy-dev-issue at codespeak.net>
<https://codespeak.net/issue/pypy-dev/issue614>
_______________________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pypy-1.4.1-add-LIBRARY_INSTALLATION_PATH.patch
Type: text/x-patch
Size: 542 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-issue/attachments/20101223/f1dda35b/attachment.bin>
More information about the Pypy-issue
mailing list